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Preface 


The Psychology, Learning, and Technology Conference (PLT) aims at becoming a 
meeting point for researchers in education, psychology, and computing, along with 
anyone who has an interest in the future of learning with technology. 

The first edition, PLT 2022, was held during January 19-21, 2022, in blended 
mode due to the COVID-19 pandemic situation. The conference was organized by the 
University of Foggia. 

The objective of PLT is to advance scientific knowledge and methodologies of 
intervention at the intersection of education and psychology that can be purposefully 
applied to the design and development of technologies that support the learning process. 
PLT 2022 engaged researchers, practitioners, educational developers, and entrepreneurs 
to address current challenges and advances in the field. 

This first edition of PLT was free, open to all, and early career researchers were 
encouraged to apply. Indeed, PLT aims to trigger and promote a community of 
practice in this area, offering different perspectives from the multidisciplinary field 
(e.g., psychology, education, computer science). 

In keeping with current times, PLT 2022 considered themes relating to the 
COVID-19 experience. In particular, the conference focused on studies that consoli- 
dated the results that emerged during the pandemic. These studies describe the data 
and approaches that face the new challenges resulting from the COVID-19 learning 
revolution. Five studies submitted to the conference explicitly featured the word 
‘COVID-19’ or ‘pandemic’ in the article title, although many of the studies were linked 
to this issue. Despite the initial period, where the shock of the pandemic produced a direct 
impact, the studies submitted show a more complex analysis of the consequences and, 
moreover, have considered the solutions and offer proposals for facing this challenge. 

In total, 23 articles were submitted to PLT 2022 and 70 authors were involved 
in the papers. Among the 23 papers submitted for peer review, 13 papers were 
accepted for this volume, leading to an acceptance rate of 57%. The authors of 
the selected papers come from seven different countries (Italy, Poland, the Czech 
Republic, Spain, Nigeria, India, and Turkey). Each paper was reviewed by three 
external reviewers. Reviewers were not allocated any papers by authors belonging to 
the same university or having any kind of pre-existing connection to avoid conflicts 
of interests; papers coauthored by members of the conference committees were also 
handled in an appropriate review process. The Program Committee included 15 
professors and researchers belonging to 10 different higher education institutions. In 
order to fulfill the assignment of three reviews for each article, we were grateful to have 
14 additional reviewers. 

An invited paper by Lucia Mason, professor of Developmental and Educational 
Psychology at the University of Padua, is also included in this book. 

As it was the first PLT event, the chairs and the University of Foggia decided to 
organize the conference as an open event. The organizers did not ask for any registration 
fee related to the conference participation, considering the open results as important for 
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the development of the scientific research. In addition, the chairs decided to open the 
conference to ‘external’ participants. The idea was to disseminate the results beyond the 
traditional academic borders. This approach was a success: many participants attended 
the conference online through their universities. In particular, many teachers and people 
enrolled in initial teacher training courses attended the conferences sessions. In the 
YouTube link used for the event, the conference obtained a peak number of about 200 
participants. 

PLT 2022 was held over three days, with sessions that covered a half-day. This book 
collects the accepted papers and it is divided into sections in line with the sessions that 
took place during the three days of the conference. 

The first section “Invited Paper” is reserved for the contribution of Lucia Mason. 

The second section, called “New Intersections Between Psychology, Learning, and 
Technology”, contains four papers reporting results from the field of psychology, ranging 
from grief as a consequence of COVID-19 to narrative practices for digital storytelling. 

In the third section, the chairs gathered all the papers linked to promoting learning 
and teaching solutions for the school sector, calling the session “Apps and Innova- 
tive Tools: New Opportunities for School Learning/Teaching”. This part contains five 
papers proposing new technologies and studies in the education field, from robotics to 
educational applications. 

Finally, the fourth section, called “Empowering Study Attitude”, collects papers in 
the field of developmental psychology, considering themes like well-being, orientation, 
and job crafting. 
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Abstract. This study examines the effectiveness of two short-term instructional 
interventions on the evaluation of digital sources in the classroom context. One 
hundred sixty-one gii graders were involved in one of three conditions: an inter- 
vention based on providing declarative knowledge (DK) on source evaluation, 
an intervention based on providing contrasting cases (CC) of source evaluation 
strategies, and a control condition (C) with no intervention. Students in both inter- 
vention conditions carried out a series of activities in two lessons. Prior topic 
knowledge and interest, reading comprehension, working memory, and perceived 
competence in online information search and evaluation were used as control vari- 
ables to ensure the equivalence of participants across conditions. In all conditions, 
participants were asked to read four documents on the debated issue of the poten- 
tial health risks associated with the use of the mobile phone and, after reading, 
to rank-order them for reliability, providing justifications for their rank-ordering, 
and to write a short essay to judge those health risks, based on the texts read. The 
findings indicate that both interventions were effective, to some extent, in promot- 
ing the use of source characteristics in reliability judgments, as revealed by the 
justifications provided by the students for motivating their rank-ordering of the 
read documents. Both interventions were also effective in supporting multiple-text 
comprehension as revealed by the students’ argumentation in short essays. 


Keywords: Source evaluation - Sourcing skills - Multiple-text comprehension - 
Digital reading - Internet 


1 Introduction 


One essential aspect of the democratization of our current societies is access to a dis- 
parate body of information through the Web. Although digital inequalities still exist 
even amongst young people, as the Covid-19 pandemic has revealed [1], in the digital 
era it is not necessary to possess books or to visit libraries to know more on unfamiliar 
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topics. The Internet has become ‘the’ source of information, being easily accessed by 
students who are almost always connected through a smartphone, tablet, or laptop. From 
lower-secondary school onwards, students are daily ‘consumers’ of online information 
and use it to carry out school assignments. Even though the ‘Google generation’ of 
students has been raised in the Internet era, their information literacy has not improved 
as a result of the wide access to technology [2]. Even older and ‘Internet savy’ students 
have difficulties to identify fake news during online reasoning [3]. It is, therefore, vital 
that we teach beginner seekers of online information how to distinguish between reliable 
and unreliable sources. The huge array of documents that they can access by a simple 
click pose new challenges to them. Even if they are able to locate and select relevant 
webpages — which may not always be the case — students must then be able to evaluate 
their credibility or authoritativeness according to appropriate criteria [4—6]. 

The term ‘sourcing’ is used to refer to readers’ use of information about the author, 
genre, and date of publication [7]. Digital literacy necessarily includes sourcing skills. 
What do we have to believe in? Why? How can this information be reconciled with the 
others? These are crucial questions to deal with when reading multiple online pages that 
focus on the same content, not only to be able to discern the sources that deserve to be 
followed up, but, more importantly, to understand source content better [8]. Sourcing 
is associated with deep multiple-document comprehension and this link is particularly 
important. Students who are more able to process a webpage for both its content and 
‘metadata’ regarding the authoritativeness of the source, are also those who are more 
able to comprehend conflicting information [9, 10]. 

Research has shown that primary [e.g., 11] and secondary school students [e.g., 12— 
13], as well as undergraduates [14] may not evaluate sources at all, or appeal to naive 
criteria when judging them and their content. Recent studies with adolescents have 
indicated that they use source information very little [15], or do not discern documents 
on the basis of author competence even when prompted with specific questions [16]. 

It is, therefore, not surprising if interventions on sourcing skills have been imple- 
mented in primary and secondary schools, as a recent review illustrates [7]. Some of 
them are long-term interventions that have not examined multiple-text comprehension 
as one of their possible effects [17, 18]. 

Our investigation focused on short-term interventions as they can be more easily 
implemented in the natural context of the classroom and also embedded in curriculum 
units of a subject like science or history. The study aimed to compare the effectiveness of 
two short-term instructional interventions implemented in lower-secondary school for 
promoting both evaluation skills of digital information sources and multiple-text com- 
prehension. One intervention provided students with essential declarative knowledge on 
how to evaluate the reliability of online sources and the accuracy of the information 
[10, 19]. The other provided two contrasting cases of source evaluation strategies where 
students were asked to evaluate which was the best and should be followed up [20]. 


2 Evaluation of Source Credibility and Multiple-document 
Comprehension 


In framing our study, we took into consideration the Discrepancy-Induced Source Com- 
prehension (D-ISC) model [21, 22]. According to this model, readers pay more attention 
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to “who says what” when they are faced with conflicting information on the same issue 
or questions that are presented by a different source. The perception of discrepancies 
between texts acts as a potential mechanism through which readers are prompted to 
process and evaluate source information more deeply when encoding the links between 
sources and the related information. Greater encoding is likely to occur when the mental 
representation of information of texts includes the links between sources and their con- 
tent and conflicting information is more likely to be integrated into a coherent overall 
mental model [23]. Not surprisingly, single text comprehension is a widely investigated 
research area, given the relevance of the skill to construct meaning from text [e.g., 24, 25]. 
Much research is based on the well-known Kintsch’s model [26]. Compared to single- 
text comprehension in this model, multiple-text comprehension includes the additional 
layer of the intertext model [27]. When reading conflicting documents, readers may 
achieve overall coherence through the intertext model, despite there being contrasting 
information. 

The link between sourcing and multiple-document comprehension is not only theo- 
retically justified, but also empirically documented [4, 9, 28, 29]. All studies revealed a 
positive relation between sourcing and multiple-text comprehension as readers’ attention 
to source information is associated with a greater comprehension of multiple conflicting 
documents. 


3 Types of Short-Term Intervention for Promoting Sourcing Skills 
and Multiple-document Comprehension 


Two main types of short-term intervention can be identified in the current literature as 
effective in promoting both source evaluation and comprehension of multiple documents 
in secondary school students. 


3.1 Intervention Based on Declarative Knowledge 


One type of short-term intervention focuses on providing declarative knowledge about 
source evaluation. An example of this type of intervention is the study by Mason et al. 
[19]. Ninth graders were given three pages of information about how to evaluate the 
reliability of a website and the accuracy of its contents. The written material included the 
“SEEK” (Source, Evaluation, Explanation, Knowledge) [10]. This material illustrated 
three main criteria for evaluating how reliable or credible websites are in the form of three 
questions: Who is the author? How reliable is the information? How well does the site 
explain the information? Readers were instructed to evaluate: for the Source (author), 
whether the authors were knowledgeable about the topic and their motivation; for the 
Evaluation of information, whether it was based on scientific evidence and whether 
similar information was provided across credible sources; for the Explanation of the 
information, whether they understood what the site said about the topic and whether 
the explanation corresponded with the scientific knowledge that they might have had, or 
with information given by other credible sources [10]. 

Participants in this study had the opportunity to practice source evaluation in a 
basic inquiry task on a given topic. A worksheet was given to them, which included 
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questions to prompt them to use the SEEK criteria in source evaluation. Next, they 
had to apply source evaluation to the transfer inquiry task on a different topic. Results 
revealed that participants who had received the instructional material with declarative 
knowledge about source evaluation in the transfer task outperformed non-instructed 
participants when rank-ordering the webpages and justifying their ranking in the transfer 
task. Moreover, SEEK-instructed participants were better at both surface and deeper 
comprehension of the conflicting information. 


3.2 Intervention Based on Contrasting Cases 


The other type of effective intervention on source evaluation is focused on providing 
contrasting cases. Contrasting cases is a classroom-based instructional practice that is 
used to promote the acquisition of declarative or procedural knowledge. Pertinent to our 
study is the investigation carried out by Braasch et al. [20] that focuses on the effec- 
tiveness of contrasting cases regarding protocols about source evaluation strategies. One 
was a protocol of expert strategies that rely on advanced criteria for judging a source; the 
other is the protocol of non-expert strategies that appeal to low-level criteria, which are 
often adopted by naive readers. Participants in the last year of upper secondary school 
were asked to decide which were the best strategies to evaluate multiple documents 
retrieved from the Internet, and to explain why. They received the instructional material 
including two to-be-contrasted student strategy protocols attributed to student A and 
student B for practice on a basic topic. For each strategy protocol, the more compe- 
tent and more critical student expressed more sophisticated strategies regarding source 
evaluation, taking into account author, venue, type, and date of publication: “...I start 
with the authors to see whether they are knowledgeable about the topic” [20, p. 184]. 
In contrast, the poorer, less critical student exhibited low-level strategies that did not 
take into consideration source characteristics: “Since this title contains my key words, I 
know I can trust and use the information” (p. 184). Results confirmed the effectiveness 
of the intervention in supporting students’ rank-ordering and justifying their ranking by 
appealing to essential features of the sources. Moreover, the intervention favored the 
inclusion of more correct scientific concepts from the more useful texts when writing 
essays about the topic. 


4 The Present Study 


Current research indicates that students need to be instructed on the important criteria to 
use for evaluating the sources and information accessed on the Web. These criteria referto 
the qualities that make a source reliable (i.e., expertise and unbiasedness) and information 
accurate (i.e., based on scientific evidence and corroborated by information from other 
reliable sites). Research also shows that relatively short-term instructional interventions 
on source evaluation, like those reviewed in the previous sections, can be effectively 
implemented in the context of upper-secondary school, improving multiple-document 
comprehension as well. 

To extend current research, we tested the effectiveness of the two aforementioned 
types of short-term instructional interventions as we do not know if the two types of 
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intervention are equally effective or if one is superior to the other, in particular in the case 
of lower-secondary school students. We focused on a two-lesson intervention, instead of 
long-lasting interventions, because the former is more easily embedded in a curriculum 
and implemented in the natural classroom context. We involved younger adolescents who 
have started to be daily seekers of online information for various purposes, including 
school assignments. They need to be competent users of Internet-based information to 
follow up only reliable sources and construct knowledge from accurate information. As 
the participants were younger than those involved in the Braasch et al. [20] and Mason 
et al. [19] studies, the interventions took place in two sessions, rather than one, to give 
them more time to practice source evaluation on basic tasks. 

The following research questions (RQ) guided the study: 

(RQ1) Are two interventions — one based on declarative knowledge on source eval- 
uation and the other on contrasting cases of source evaluation — equally effective in 
promoting students’ reliability judgments of various online documents? 

(RQ2) Are the two interventions also equally effective in supporting multiple- 
document comprehension? 

Basedon the extant literature [19, 20], forRQ1, we hypothesized that students in both 
intervention conditions would express greater source evaluation skills than students in 
the control condition. Specifically, we expected that both intervention conditions would 
promote more reliability judgments based on source characteristics compared to the 
control condition for both high and low reliability documents (Hypothesis 1a). In addi- 
tion, we expected a difference between the two intervention conditions in favor of the 
contrasting cases (Hypothesis 1b). The reason for this is that they acknowledge and 
target inappropriate source evaluation strategies often adopted by students and make 
what should and should not be considered in source reliability judgments more salient. 
In other words, the comparison/contrast process gives learners the opportunities to form 
a more differentiated knowledge base, which promotes the identification and interpre- 
tation of salient features in novel contexts [20]. On the other hand, compared to the 
control conditions, participants in the declarative knowledge condition would benefit 
from explicit instructions on what to consider in source evaluation for reliability. How- 
ever, this advantage would be inferior to that promoted by the contrasting case situation, 
as the students would not be confronted with two concrete reliability judgments. 

For RQ2, we considered that the link between source evaluation and multiple- 
document comprehension is not only theoretically legitimate [21], but also empirically 
documented [4, 9, 28]. Our hypothesis, therefore, was that students in both intervention 
conditions would also outperform those in the control condition for multiple-text com- 
prehension (Hypothesis 2a), with a superior advantage for those who were provided with 
contrasting cases of source evaluation for reliability (Hypothesis 2b). This is because 
they would scrutinize source features and it would be easier for them to reconcile con- 
trasting information in an integrated representation of multiple documents on the same 
topic [20]. 
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5 Method 


5.1 Participants and Design 


One hundred sixty-one 8th graders (Mage = 13.44, SD = 0.7; girls = 79) from two lower- 
secondary schools participated in the study and were assigned to one of three conditions: 
intervention on declarative knowledge (DK, n = 51), intervention on contrasting cases 
(CC, n = 58), and no intervention or control (C, n = 52). Each condition included 
three classrooms in total (from both schools). All participants were native-born Italian 
and shared a middle-class socio-economic status. They voluntarily participated upon 
written parental consent. For practical constraints relating to school organization and 
classroom management, we used a quasi-experimental design with random assignment 
to conditions at the level of intact classrooms. 


5.2 Interventions on Source Evaluation 


Both interventions took place in two sessions, with the first about genetically modified 
food and the second about dinosaur extinction. At the beginning of the first session of 
both interventions, for 5 min the instructor introduced the issue that the Internet provides 
many types of document and readers must consider their reliability in order to understand 
the contents well. To reduce researcher bias, a scripted session plan was prepared for 
each of the intervention conditions and the control condition. 


DK Intervention Condition. After the 5-min introduction, in this intervention condi- 
tion students were informed that they would read material regarding source reliability. 
They were then provided with three pages of declarative information about how to eval- 
uate the reliability of a website and the veracity of its content. The material was taken 
from Wiley et al.’s [10] instructional unit on Source, Evaluation, Explanation, Knowl- 
edge (SEEK). The declarative material explained that three main criteria are to be used 
when evaluating the reliability of a website, specifically: (1) Who is the author? (2) 
How reliable is the information? (3) How well does the site explain the information? 
(pp. 1098-1099). 

In the first session, the documents read were taken from two websites about GM 
food, one being more reliable than the other. One document was written by a professor 
of agrarian microbiology, published in an online journal (414 words); the other was 
taken from a site of gourmet food and wellbeing (414 words). In the second session 
the documents read were about dinosaur extinction, again one being more reliable than 
the other. One was taken from an information site on dinosaurs, which was written by 
its webmaster (365 words), while the other was taken from the online bulletin of the 
national institute of astrophysics and written by a scientific journalist (365 words). 

For both topics, the order of appearance of the most and least reliable document was 
counterbalanced. In both intervention sessions, students were given a worksheet for each 
site, with questions about the author, information, and explanation. The worksheets were 
intended to support students’ practice in the use of the provided declarative information 
on what to consider when evaluating an information source for reliability. 
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CC Intervention Condition. After the 5-min introduction based on Braasch et al. [20, 
p. 186], in this intervention condition students were informed that they would read 
about the different strategies that two students from another lower-secondary school 
used to evaluate the source reliability of two documents on a specific topic. Participants 
were asked to compare and contrast the two students’ strategies to distinguish what good 
readers do differently from poor readers when learning from information on the Internet. 
Participants were also instructed to take into account that one of the two students, who 
made comments on the documents, used better strategies than the other. The aim was to 
guide students to understand why some strategies are better than others. 

In the first and second sessions, students were presented with the same two documents 
used in the DK intervention. For each topic, participants read the two documents and 
a pair of fictional students’ protocols that reported the strategies used by student A 
and student B when evaluating the documents. Participants did not receive any clues 
regarding which student adopted the better strategies. To exemplify, in the first session 
on the topic of GM food, the more critical student started by saying: “I have seen that 
the first is written by an agriculture professor who teaches at the university, while the 
second is written by gourmets. Therefore, only the first is written by a competent person, 
who is real expert in the topic”. 

In the second session on the topic of dinosaur extinction, the less critical reader started 
by saying: “I have noticed that the first text is of the online bulletin of the institute of 
astrophysics, while the second is a site about dinosaurs. I am very interested in dinosaurs, 
and I always liked to read about them and watch cartoons on these animals. The site on 
dinosaurs is easier and more interesting...I like the idea that a big asteroid has fallen on 
the earth and caused a big crater. I do not like the idea that also volcano eruptions matter”. 
The ‘better’ and ‘poorer’ students’ protocols were of the same length (208 words). 

Participants were also given a worksheet that asked to identify which student used 
the better strategies for evaluating the two documents and selecting the most reliable, as 
well as explaining as clearly as possible the reasons why they were the better strategies. 
The worksheets set out to support students’ practice in source evaluation through the 
identification of the features to be considered when evaluating an information source for 
reliability. In the last ten minutes of the session, the instructor wrote on the blackboard 
student-generated suggestions about good and poor strategies in document evaluation. 


Control Condition. Participants were provided with the same 5-min introduction as in 
the two-intervention conditions. They also read the same documents on GM food and 
dinosaur extinction during the same class periods, but no intervention on source evalu- 
ation was implemented. In other words, participants read exactly the same documents 
as those in the intervention conditions to be exposed at the same controversial topics, so 
they were not disadvantaged in terms of experience with different points of view on the 
same topic. However, the importance of source evaluation was not emphasized to them, 
nor were they provided with any specific information on source evaluation. 


5.3 Application-Task Materials and Procedure 


The material used in the application-task session, which was the same in all conditions, 
consisted of four documents read on a computer screen in the school computer lab. 
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Used in a previous study [30], the documents were taken from real sites (stored locally) 
and looked like the original; only the language was modified, in some cases, to make it 
simpler. They were balanced for reliability (low/high) and position on the debated topic 
of the potential health risks associated with the use of mobile phones. The two higher 
reliability documents were: (a) the report of an interview with a scientist, an expert in 
molecular biology, published in the science section of a newspaper and explaining the 
biological impact of radiations from mobile phones and (b) the report of a pediatrician, 
published on the site of the national association of pediatrics and stating the inconclusive 
nature of current scientific evidence, especially for children, but recommending many 
precautions. The two lower reliability documents were (c) the report of a webmaster, 
published on an online magazine on mobile phones, describing inconclusive data and 
wondering who would give up using the mobile phone and (d) the personal blog of an 
unknown supporter of natural life, describing various serious health problems caused by 
mobile phones. 

At the beginning of each text, information about the author, credentials, and date of 
publication was provided. Each text included the same number of words (424 words). 
Participants were instructed to read the documents carefully at their own pace as they 
would then be asked to complete some tasks. 


5.4 Pre-intervention Measures 


Because of the quasi-experimental design of the study with random assignment to con- 
ditions of the level of intact classrooms, some individual differences were assessed to 
ensure that participants did not differ among conditions for a number of potentially 
interfering variables. 


Topic Prior Knowledge. Prior knowledge of the application topic was measured using 
five open-ended questions about electromagnetism and the potential health risks asso- 
ciated with the use of mobile phones, taken from Bråten et al. [9]. Answers to these 
questions were analyzed for content and scored 1 for each correct information unit men- 
tioned (range: 0-3). A random selection of 80 students’ responses were scored by the 
second and third authors, resulting in an 89% agreement for all answers. Disagreements 
were resolved through discussion. The third author scored the remaining responses. 


Topic Interest. It was measured using a 10-item self-report scale about the value and 
importance of knowing more about the potential health risks associated with the use of 
mobile phones (Cronbach’s alpha = .84), examples item: “I like to be updated on the 
health consequences of the continuous use of mobile phones”. 


Reading Comprehension. It was measured using the Italian standardized test for the 
appropriate grade [31]. Participants read an informational text and answered 15 multiple- 
choice questions. One point was assigned to each correct answer. 


Working Memory. It was measured using the Italian version of the well-known Dane- 
man and Carpenter [32] Reading Span Test, which evaluates the simultaneous processing 
and storage of unrelated information and is, therefore, considered a complex span text 
[33]. 
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Perceived Competence in Online Information Search and Evaluation. It was mea- 
sured using a six-item self-report scale (Cronbach’s alpha reliability = .78). A sam- 
ple item is: “I always do well when I look for useful online information for a school 
assignment”. 


5.5 Post-intervention Measures 


Multiple-document Comprehension. It was measured with an essay task as in many 
previous studies [e.g., 9, 34, 35]. Participants were asked to write a short essay to judge 
the health risks of mobile phone use, based on the texts read. Following Braten et al. 
[9] and Mason et al. [34], the essays were scored for sourcing and argumentation. For 
sourcing, we considered (a) the total number of explicit references to the four source 
documents and (b) the total number of source-content links, that is, explicit and implicit 
references to the four source documents that also mentioned content from those sources. 
A composite score was computed for sourcing. 

For argumentation, we considered whether both perspectives were reported and jus- 
tified, and the unresolved nature of the debated issue was acknowledged. The essays 
were scored 1-3: 1 point was assigned for essays that reported only one position on 
the debated topic, with no reference to the controversy; 2 points were assigned when 
the negative and the more ‘neutral’ positions on the topic were reported, with no refer- 
ence to the ‘openness’ of the issue; 3 points were assigned when the two positions were 
reported, but also where there was a need for more scientific information. A random 
selection of 80 essays were scored by the second and third author, resulting in an overall 
90% agreement on the essays. Disagreements were resolved through discussion. The 
third author scored the remaining responses using the same coding system. 


Rank-Ordering. Participants were asked to rank-order the four documents, from the 
most to the least reliable [4], assigning the value 1 to the website judged as being more 
reliable and the value 4 to the website judged as being the least reliable. Participants were 
awarded 1 point for ranking the two most reliable sites either as no. 1 or no. 2 and 1 point 
for ranking the two least reliable sites as either no. 3 or no. 4. When rank-ordering the 
documents, readers did not have the opportunity to look back at them but were provided 
with a randomized list of the URLs names. 


Justification for Rank-Ordering. Participants were asked to motivate their rank- 
ordering, providing one or more justifications. These were analyzed for content and 
a coding system was then used, inspired by the categories identified in previous studies 
[e.g., 36]. The categories were the following: 


— source characteristics (authors’ credentials based on expertise and authoritativeness), 
for example: “This is credible and authoritative for knowledge”; 

— personal opinion (information corresponds to the reader’ opinion), for example: “I 
disagree with what the site says”; 

— reference to other sources (information is corroborated by other sites), for example: 
“Tt is the only one to say that”; 


12 L. Mason et al. 


— reference to the content of the document read (information is easier to understand or 
interesting and appealing), for example: “This is more interesting and easier”. 


For each category, the scoring was dichotomous: 1 point was assigned when a jus- 
tification explicitly mentioned the specific aspect of the category; 0 points when it was 
not mentioned. A random selection of 80 students’ responses were scored by the second 
and third authors, resulting in an 87% agreement for all answers. Disagreements were 
resolved through discussion. The third author scored the remaining responses using the 
same coding system. 


6 Results 


6.1 Preliminary Analyses 


Data were first screened for normality. Descriptive statistics showed that they did not 
substantially deviate from normality for skewness and kurtosis. AMANOVA was then 
performed to ensure that participants across conditions did not significantly differ for 
prior topic knowledge and interest, reading comprehension, working memory, and per- 
ceived competence in online information search and evaluation. The effect of condition 
did not emerge, Hotelling trace = .10, F(10, 306) = 1.70, p = .080. This outcome indi- 
cated that the participants were comparable across conditions for all these potentially 
interfering variables. To proceed in the most parsimonious way, we did not consider the 
latter in the subsequent analyses. Descriptive statistics for individual differences as a 
function of condition are reported in Table 1. 


6.2 RQI1: Effectiveness of the Intervention for Source Evaluation 


Rank-Ordering. We firstconducted an ANOVA with rank-ordering scores as the depen- 
dent variable. The effect of condition was not significant, F < 1. Scores were substantially 
similar across the DK (declarative knowledge, M = 1.37; SD = .66), CC (contrasting 
cases, M = 1.38; SD = .67), and control (no intervention, M = 1.29; SD = .57) conditions. 


Justifications for Rank-Ordering. We also qualitatively examined the justifications 
provided by participants to appropriately rank-order each of the four documents. Tables 2 
reports frequencies and percentages of justifications for accurately rank-ordering the 
four documents as a function of condition. Non-parametric tests, specifically Kruskal- 
Wallis and Mann-Whitney, were then carried out to see whether statistically significant 
differences would emerge across categories and as a function of condition. As a measure 
of the effect size, for Kruskal-Wallis tests we used epsilon square (e°) and for Mann- 
Whitney tests we used r [37, 38]. 

Justifications for Rank-Ordering Document No. 1. We examined whether there were 
statistically significant differences across conditions for the criteria used by participants 
to support their evaluation of a document as the most reliable. First, a Kruskal-Wallis 
test was carried out with Bonferroni correction applied, resulting in a significance level 
set at p < .012, given 4 pairwise comparisons (response categories). This test revealed 
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Table 1. Descriptive statistics of all pre-intervention variables by condition 


Condition 
DK CC C 
(n = 51) | (n = 58) | (n = 52) 
Prior topic knowledge M 1.98 1.93 1.54 
SD 1.40 1.47 1.14 
Skewness | 17 .60 .26 
Kurtosis 1.02 .65 53 
Prior topic interest M 28.20 29.57 29.46 
SD 8.51 8.09 7.32 
Skewness | .38 .16 —.24 
Kurtosis 46 .34 37 
Reading comprehension M 9.10 9.03 8.23 
SD 2.94 2.58 2:32 
Skewness | —.38 31 .01 
Kurtosis .63 71 24 
Working memory M 1.94 1.66 1.88 
SD .64 .63 .70 
Skewness | .51 .86 .S1 
Kurtosis | 1.25 1.77 40 
Perceived competence in online information search | M 20.59 19.71 20.63 
and evaluation SD 3.95 4.34 4.31 
Skewness | —.46 .26 .84 
Kurtosis | .74 —.02 1.25 


Note. DK = declarative knowledge, CC = contrasting cases, C = control 


significant differences for using judgments based on source characteristics [y? (2) = 
22.39, p < .001, £? = .20] and its content [x7 (2) = 10.08, p < .006, £? = .09]. Mann- 
Whitney U tests were then performed with Bonferroni correction applied, resulting in 
a significance level of .016, given 3 pairwise comparisons (conditions). As concerns 
the use of source characteristics for reliability judgments, participants in both DK (U 
= 345.00, p < .001, r = .42) and CC (U = 293.00, p = .009, r = .39) conditions 
outperformed those in the control condition, while the two intervention conditions did 
not differ from one other. As regards the reference to source content, students in the 
control condition used more this criterion than those in the DK (U = 394.00, p = .002, 
r = 42) and CC conditions (U = 484.00, p = .009, r = .25), while the two intervention 
conditions did not differ from one other. 


Justifications for Rank-Ordering Document No. 2. A Kruskal-Wallis test did not reveal 
significant differences across conditions for the justifications provided by the participants 
to judge a document as the second more reliable among the four webpages. 

Justifications for Rank-Ordering Document No. 3. A Kruskal-Wallis test revealed sig- 
nificant differences among the participants for using judgments based on source char- 
acteristics x? (2) = 8.82, p = .010, £? = .09], and based on the content provided x? 


14 L. Mason et al. 


(2) = 18.95, p < .001, £? = .20]. Mann-Whitney U tests showed that only participants 
in the CC condition used the crucial criterion of source characteristics more frequently 
than participants in the control condition (U = 275.00, p = .002, r = .29). Reference to 
document content in the reliability judgments was more frequent in the control condition 
than among participants in both DK (U = 261.00, p = .002, r = .30) and CC conditions 
(U = 200.00, p < .001, r = .41), while the intervention conditions did not differ from 
one other. 

Justifications for Rank-Ordering Document No. 4. A Kruskal-Wallis test showed sig- 
nificant differences across conditions for participants’ judgments based on source char- 
acteristics x? (2) = 14.72, p = .001, e = .12]. Mann-Whitney U tests revealed that 
students in both DK (U = 544.00, p < .001, r = .35) and CC conditions (U = 648.00, 
p = .001, r = .31) took this crucial criterion more often into consideration than their 
counterparts in the control condition. 


6.3 RQ2: Effectiveness of the Intervention for Multiple-document 
Comprehension 


We performed a MANOVA with the scores for sourcing and argumentation, as reflected in 
the essay task, the dependent variables being, overall, positively correlated (r = .20, p = 
.009). The MANOVA revealed an overall small significant effect of condition, Hotelling 
trace = .06, F(4, 312) = 2.51, p < .041, né; = .03. Follow-up univariate analyses of 
variance indicated that differences emerged for argumentation, F(2, 158) = 4.80, p = 
.009, np = .06, but not for sourcing, F (2, 158) = .75, p = .471. Pairwise comparisons 
with Bonferroni correction revealed that both students in DK (declarative knowledge, p= 
.017, 95% CI [—.65, —.04]) and CC (contrasting cases, p = .033, 95% CI [—.61, —.01]) 
conditions outperformed students in the control condition. Participants who received 
an intervention on source evaluation, regardless of its type, generated greater arguments 
compared to their counterparts who were not exposed to any intervention. Table 3 reports 
the scores for sourcing and argumentation as a function of condition. 


7 Discussion 


Our first research question (RQ1) asked whether two classroom interventions are equally 
effective in promoting students’ reliability judgments of various online documents. In 
accordance with hypothesis la, the analysis of the justifications provided to motivate the 
rank-ordering of the four sites revealed that the students in both intervention conditions 
were more able to appeal to the characteristics of the source (expertise and authoritative- 
ness) than students in the control condition when appropriately judging the most (no. 
1) and least (no. 4) reliable site, while the two intervention conditions did not differ- 
entiate. For the second least reliable (no. 3) site, only students in the CC intervention 
condition considered more the quality of the source than those who did not receive any 
intervention. 

When differences emerged for appealing to personal opinions, they indicated that 
the latter students used this criterion more than their counterparts in the intervention 
conditions. Such criterion can lead to myside or confirmation bias, that is, to judge a 
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Table 2. Frequencies and percentages of justifications for rank-ordering the four online docu- 


ments 


No. 1: Interview with a scientist 


Condition 

DK CC C 

(n= 51) (n = 58) (n = 52) 
Type of justification f % n % n % 
Source characteristics 29 76.3 27 64.3 15.6 
Personal opinion 1 2.6 - - 2 6.3 
Corroboration across sources - - ni — 9.4 
Source content 20 52.6 25 59.5 28 87.5 
No. 2: Italian society of pediatrics 
Source characteristics 19 54.3 19 50.0 10 27.02 
Personal opinion 11.4 13.2 8 21.6 
Corroboration across sources 11.4 5.3 1 2.7 
Source content 20 57.1 19 50.0 23 62.2 
No. 3: Online magazine 
Source characteristics 16 48.4 20 57.1 19.2 
Personal opinion 4 12.1 5 14.3 4 15.4 
Corroboration across sources 21.2 1 2.9 1 3.8 
Source content 15 45.5 10 28.6 22 84.6 
No. 4: Personal blog 
Source characteristics 22 55.0 22 48.9 14.0 
Personal opinion 3 7.5 9 20.0 20.9 
Corroboration across sources 5 12.5 1 22 11.6 
Source content 24 60.0 22 48.9 32 74.4 


Note. DK = declarative knowledge, CC = contrasting cases, C = control. Students could refer to 


more than one criterion to justify their ranking. Percentages are computed on the total number of 


justifications given by the participants 


document as reliable or unreliable regardless of the real authoritativeness of the web 
source, but only because it is aligned (or not) with the reader’s own opinion on a con- 
troversial topic [39]. It is worth noting that very few students in all conditions appealed 
to corroboration across sources to justify their rank-ordering. A plausible explanation is 
that there were few documents available, and the same position on the topic was shared 
by only two online resources, one of which was more reliable and the other less so. Thus, 
this important aspect for epistemic source evaluation my have not been salient enough 


in the current study. 
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Table 3. Descriptive statistics for post-reading variables of sourcing and argumentation by 
condition 


Condition 
DK CC C 
(n = 51) (n = 58) (n = 52) 
Sourcing M 2.14 1.91 1.71 
SD 1.96 1.75 1.53 
Skewness 1.27 .80 1.18 
Kurtosis 1.25 .07 1.61 
Argumentation M 1.31 1.28 .96 
SD .67 .61 .62 
Skewness 1.52 1.63 52 
Kurtosis 1.65 2.37 1.55 


Note. DK = declarative knowledge, CC = contrasting cases, C = control 


The two types of short-term intervention did not differ in rank-ordering and from 
the control condition. This outcome is not aligned with Mason et al. [19] intervention 
study that involved upper-secondary school students providing declarative knowledge 
about source evaluation. However, in that study 9th graders were asked to rank-order 
nine documents, while in the current study there were only four documents to be rank- 
ordered by 8th graders. A possible explanation for the lack of significant differences 
across conditions in the current study is that the task may have been too easy due to 
the lower number of documents. An alternative explanation is that the documents were 
easy to discriminate, even by 8th graders, who could perceive their different levels 
of reliability. Recent research on source evaluation has documented low rank-ordering 
skills in 7th graders when they were provided with six online documents [33]. Given 
the mixed results in the literature, in future research both the number of documents and 
degree of differentiation for source reliability should be taken into account when using 
a more challenging set of materials to be evaluated by adolescents. 

The finding about the lack of differences for rank-ordering is also in contrast with 
the results of the Braasch et al. [20] study that involved 13th graders, who were provided 
with two contrasting cases of source evaluation. However, in the Braasch et al. [20] 
study, students exposed to the intervention were better able than the control students to 
discriminate six documents for usefulness, not reliability. Judgments of the utility of a 
document to learn about a topic are easier to make than judgments of the reliability of 
a source, as the former are strictly related to the content provided. In contrast, the latter 
refer to the ‘metadata’ regarding author credentials, which may not receive particular 
attention during document processing, as indicated by studies with older students [40], 
even when they are explicitly asked to evaluate the information about sources [41], or 
with middle school students, even when they are specifically prompted to consider source 
competence [16]. 

Our second research question (RQ2) asked whether the two interventions were also 
equally effective in supporting multiple-document comprehension. Results confirmed 
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hypothesis 2a as both intervention conditions led to greater argumentation about the 
controversial topic than the control condition. This finding is aligned with previous 
research on high-school students’ multiple-text comprehension after a short [19] and 
long-term source intervention [17]. Readers’ essays did not differentiate as a function 
of condition for sourcing (number of sources and source-content links). This outcome 
may be related to the previous outcome on rank-ordering. As students were equally 
able across conditions to discriminate the online sources, although not always appealing 
to advanced criteria, in their written essays they did not differentiate for citing and 
relating the sources to their contents, but they did for forming a coherent and integrated 
representation of the contrasting information. 

For both research questions, contrary to our hypotheses 1b and 2b, a superior advan- 
tage of providing contrasting cases over declarative knowledge did not emerge. The 
mechanisms underlying the two interventions are different. In the direct, more ‘tradi- 
tional’, prescriptive intervention students bear in mind what to do and try to apply it. In 
the contrasting cases intervention, students are stimulated to solve an information prob- 
lem, scrutinizing and identifying the best of two strategies without any direct prompt. 
Either one or the other mechanism has potential for sustaining, at least to some extent, 
sourcing and comprehension of conflicting information in eighth graders. 

To sum up, the findings of this study do not allow us to determine if one of the two 
short-term interventions on source evaluation should be preferred to the other. By either 
allowing students to know directly what to look at, or stimulating reflections on different 
evaluation strategies, a consideration of source characteristics and a comprehension 
of multiple documents were encouraged. What we can expand from the current data, 
therefore, is that the two types of short-term intervention can be implemented in the real 
context of lower-secondary school classrooms and are substantially effective, at least to 
some extent and in a short time frame. 


8 Limitations 


Some limitations should be taken into account when interpreting the results of the current 
study. First, although both interventions favored reliability judgments based on digital 
source characteristics and multiple-text comprehension compared with a control condi- 
tion, the effect sizes are small. Future research with more participants in each condition 
may provide more solid results. Second, we did not use process data to understand the 
underlying mechanisms that can explain the benefits of each type of intervention. Studies 
based on eye-tracking methodology, for example, can reveal if and where the interven- 
tions induce greater allocation of attention while reading the various materials provided 
to support source evaluation. The analysis of process and outcome findings will allow 
an in-depth understanding of the link between reading behavior and offline sourcing 
and argumentation. Third, it remains open to future studies if, and to what extent, what 
students learned during the intervention is also long-lasting, and therefore used over 
time when they spontaneously retrieve information, evaluate, and comprehend online 
documents for inquiry purposes. 
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9 Conclusions 


Despite these limitations, the study has significance as it expands the evidence that two 
types of short-term intervention on source evaluation, which can be easily implemented 
in the classroom, are to some extent effective. The effectiveness of the interventions 
emerges in supporting students to appeal to the epistemic characteristics of information 
sources when judging the reliability of digital documents, as well as in constructing 
an integrated and comprehensive representation of multiple documents. Overall, the 
study indirectly contributes to the debate on the degree of guidance that promotes better 
learning, supporting the idea that either direct or guided instruction works [42]. 

Practical implications can also be drawn from the study as the critical use of contem- 
porary digital media emphasizes the need to support students in becoming thinkers who 
are epistemically vigilant [43] and able to reason well in an information-saturated world 
steeped in socio-scientific issues that matter greatly in our society [44]. If students are 
equipped with source evaluation skills to be critical consumers of information, they will 
more likely be future citizens able to act as rational thinkers in decision-making pro- 
cesses that impact the individual and societal life. The results highlight the importance 
and feasibility of providing students with source evaluation skills from the early years 
of secondary education. Moreover, alternating one and the other kind of intervention 
to sustain sourcing when dealing with complex socio-scientific issues may represent a 
potentially effective pedagogical strategy. It combines direct and indirect instruction, 
depending on situational contexts, in the service of students’ involvement and cognitive 
performance in a crucial task in the Google era. 
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Abstract. Around midmonth March 2020, The World Health Organization 
acknowledged the COVID outbreak as a “public health emergency of interna- 
tional concern.” Worldwide, various heads of state-imposed mandatory lockdowns 
to curb and ease the spread of the disease. With such restrictions in hand, the 
education sector was among the many which felt the pinch brought about by the 
restriction measures imposed. To try and flatten the curve and minimize the spread- 
ing of the virus from one person to another, there was a need for reduced physical 
contact among people. According to Shahzad et al. (2020), public gatherings such 
as parties, religious services, social amenities, political gatherings, and physical 
learning for both universities and lower levels stopped since health comes first 
before anything else. With that in hand, the world had to look for suitable solu- 
tions to cope with the pandemic since the virus was not ending time soon; as a 
result, the globe adapted ‘the new normal of life. In response, education facilities 
had to embark on e-learning since life had to continue, which is great gratitude to 
technology. However, it brought about various issues that we will try and highlight 
since a change in anything is always accompanied by multiple challenges. 


Keywords: Motivation - Distance learning - Universities - Covid-19 - Review 


1 Introduction 


Technology has changed traditional education to the current form of accessing educa- 
tion, an example being artificial intelligence. Shahzah et al. (2020) define E-learning 
as technology-based education retrieved through websites, learning thresholds, video 
conferencing, YouTube, mobile applications, and other various platforms through avail- 
able websites of melded education tools. Also, virtual learning is sharpening scholars’ 
understanding, also the educational staff and professional and industry people abilities 
via the internet. Currently, a majority of the complex learning universities are offering 
virtual programs for their scholars both around and off campuses, making education 
accessible to everyone despite your location, provided you have a laptop or mobile 
device with internet connectivity. In the current world, the eminence of teaching and 
excellent infrastructures such as computers and IT up-to-date apparatus reception are 
now in great ultimatum. Higher-level education facilities such as universities are shifting 
their schooling designs with the use of intelligent capital. Shahzah et al. (2020) explain 
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that universities closed down due to the COVID-19 epidemic, and most lecturers and 
students were happy to shift to online education. However, the sudden change from face- 
to-face learning to virtual education was faced with various challenges that hindered the 
smooth and fluent submission of learning resources among learning institutions. 

The online tutelage milieu comprises of two parts of intermingling styles: the scholar, 
trainer, and content, and the second comprises of know-hows and approaches of inter- 
linking. Shahzah et al. (2020) explain that Covid-19 positively and harmfully impacted 
the online education aspect of learners’ lives to a temperate level. Improving social lives 
among students in online education is believed to be the core benefit of online learning 
all through the restriction period. 


1.1 Effect of Covid-19 on University Education 


In the current times, virtual learning constructs a more noteworthy impression on all 
kinds of students, as much as the freelance and permanent in the higher education insti- 
tutions, and it is becoming very helpful to students since a majority of them are work- 
ing in companies easing their schedules due to time constraints (Shahzah et al. 2020). 
COVID-19 theatrically restructured the way worldwide education is conveyed. Sums of 
learners were impacted by educational institutions’ closure due to the endemic that has 
led to the greatest’ shipment’ in the history of education. Imposing a 24-h restriction 
changed students’ lives, specifically at home, where they spent significant amounts of 
time. The results of this research show that remaining at household assisted students 
clearly restructure their priorities. The retro was a good time for spiritual replications 
and gaining equanimity, and enabling them to enjoy more time with their families. Feel- 
ings of solitude and being away from their communal associates, on the other side, 
were confirmed as negative impacts, implying that a passionate aspect of scholars’ life 
lacked throughout their curfew experience. As a precaution, all venues where daily social 
activities take place have shut. 

With the pandemic, all university facilities have been faced with various psycho- 
logical, physical, and economic challenges, resulting in disorders in the running of 
these facilities. Browning et al. (2021) explain that university students are increasingly 
renowned as an exposed populace, going through higher levels of fretfulness, despair, 
drug abuse, and disorder eating equated to the general populace. Therefore, with the 
drastic change in the nature of their educational familiarity, changes such as sheltering 
in place throughout the corona plague created a more significant encumbrance on the stu- 
dents’ mental health. Based on research conducted in the United States, college students 
contributed immensely to the population of the citizens stalwartly hit by COVID-19 due 
to ambiguity regarding academic progress, personal careers, and their societal life during 
their college life (Browning et al. 2021). Previously, we knew that university students 
rarely had psychiatric and counseling services, which is critical and essential to warrant 
immediate health interventions that would have helped curb the issue among students. 
However, according to (Browning et al. 2021), generally, U.S. students seemed not to 
have been affected toughly academically by the COVID-19 pandemic as elsewhere in 
the world. As a matter of fact, they portrayed a normal satisfaction with most of the sur- 
vey elements. Moreover, compared to other parts of the world, American students were 
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among the first to expose their confidence in having competent computer literacy such as 
attending online classes and satisfaction with the support of the teaching practitioners. 
The pandemic crisis substantiated the concerns of international institutions, an exam- 
ple being the United Nations, which stresses the prominence of the resourceful delivery 
of educational curriculums to shun from digital and social inequalities. Cessation of 
movement among citizens that required people to stay at home the whole day while also 
going on with the studies under unfamiliar environments had an impact among univer- 
sity students interpersonal and intrapersonal live in both pedagogically and publically 
(Alghamdi 2021). Another significant challenge posed on higher education learning is 
the students living accommodation. Almost all universities globally offer accommoda- 
tion to students, even if not to all due to the capacity differences. However, with the 
enlisted health guidelines, it was difficult for university students to accommodate stu- 
dents with the previous capacity before the pandemic in their dormitories. Hence it posed 
a greater traumatizing scene to most students who never got dormitory slots within the 
school and had to rent with the school environs, and it was pretty expensive for them. 
Younger students tend to experience greater fear since students in the age bracket 
of 18 and 24, irrespective of their educational grade, tend to be more angst about their 
educational forthcoming and their capability to pay for their university learning equated 
to older students. Also, due to the barrage of Covid-19 news, younger students who are 
addicts to societal media platforms get exposed to risk texts compared to older scholars. 
More than a hundred partakers were enlisted cross-sectional from the targeted sam- 
ple at seven significant state universities in spring. From the large population invited 
to play a part in the survey, 2,534 reactions with data on a majority of the relevant 
variable. According to Browning et al. (2021), there is anticipation that the assessment 
would be tough to tight-fistedly and systematically capture the broad array of impact 
from the COVID-19 on scholars with qualitative measures. Also, there was an unre- 
stricted questionnaire that enquired respondents how the pandemic changed them and 
how they were feeling and what was on their minds. The current study’s corresponding 
author conducted unstructured personal interviews about their early experiences with 
the COVID-19 epidemic in terms of new interview data. In February 2020, ten individ- 
uals aged 18 and over got gathered for these interviews. Enrollment happened in low- 
and high-risk areas across the America, including Washington’s metropolitan areas and 
countryside areas in Tennessee, Iowa, and South Carolina. During the pandemic, the 
dialogues taken the outlooks of the interviewees. 
E-learning. 
The educational problems that the world is facing issues with as a byproduct of the virus 
have become highly burdensome. In spite of efforts at gradual normalization in learning, 
it seems that any semblance of normal will take some time to get to be a reality. Learners, 
instructors, school administrators, and parents all got tangled up in an educational process 
in which the fundamentals of face-to-face education had to get converted to digital 
environments in a short period of time. The benevolence of information technology’s 
impact on numerous aspects of lives currently cannot be contradicted. According to 
Olum et al. (2020), e-learning has been implemented in many settings to speed up the 
furtherance of medical training. Nonetheless, the crisis led to stress among the world 
population, disrupting education globally. In order to facilitate all students so as to carry 
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on with their studies and achieve personal career goals, there was a need to implement 
online classes for students in all learning institutions. However, research conducted on 
university students across the U.S.A indicated that over half of scholars were concerned 
about the lack of social interfaces in the virtual learning environment. At the same 
time, about 10% had issues with the online learning setting (Olum et al. 2020). With the 
evolution of technology and the presence of copious parties to facilitate higher education, 
the majority of universities and educational institutions have shifted to conniving and 
hurling e-learning programs for students. (Olum et al. 2020). Various studies have shown 
that virtual learning is a beneficial tool for attaining educational requirements, expressly 
in developing nations. 

With the great potentials brought forward by digital learning, some students some- 
times opt to drop out of school (defer) and become indisposed to embark on their studies, 
which acts as a significant drawback to their literacy evolving. However, due to the pan- 
demic’s constraints, some students tend to drop out of school since they lack sufficient 
funds to continue with their classes. During the pandemic, so many people lost their 
jobs. As a result, they experienced poor financial status that could not support the access 
of internet connectivity and gadgets such as the computer to stream live classes (Abbasi 
et al. 2020). Nonetheless, as a result of the high recognition of students to enroll in 
online learning and the role of refining the eminence of education, the quality of learning 
impacted students can improve immensely. 


2 Research Methods 


2.1 Search Strategy 


This systematic review used in this analysis adopted the recommendations of Dhin- 
gra et al. (2020). A systematic review was conducted using PubMed, Science Direct, 
Google Scholar, and Bloomberg CityLab, revealing a database of studies on e-learning 
in universities during the Covid-19 pandemic. The study was narrowed down to when 
the pandemic began to now and what impact it has had on learning among universities 
globally. Additionally, the search strategy was based on identifying the keywords, for 
example, motivation, academic success, university, and covid-19. 


2.2 Study Identification 


All journals, including cohort studies, review, and case studies were eligible if they 
highlighted more than ten cases of interview on students concerning their experience 
with the new form of learning. After removing the duplicates by comparison of DOL, all 
abstracts from identified publications were examined for eligibility. 


2.3 Study Selection 


The original study search brought forward to the identification of 150 studies with com- 
plete data. However, all this was made possible through automated electronic searches 
on various database platforms. Automated search engine generated 3762 results out of 
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which settled for 150 studies that could be screened and selected the once for use in 
the case study. From the remaining studies, 20 of them were nullified since they did not 
meet the minimal requirements for the study. However, the remaining 70 were screened 
through the eligibility criteria and out of that, 21 failed to meet the eligibility criteria. 
The Fig. 1 below shows the PRISMA 2021 flow diagram for updated system reviews 
(Liberati, A. et al. 2009). 


2.4 Description of Studies 


Among the research studies, the initial study conducted an investigation in a dentistry 
college regarding their perception and opinions regarding e-learning during the lockdown 
compared to the prior physical learning. Two of the inclusive resources highlight the 
students’ motivation towards online classes in respect to internet connectivity and its 
accessibility during Covid-19 pandemic. While the third one encompasses its study on 
some of the activities and issues revolving around the capability of different students 
attending to classes, and some of the issues within United States learning process and 
third-world countries. The last two research are based on how tiresome it has become 
for students to undertake to their assignments from online classes and the challenges 
associated with practical learning courses. 


3 Results 


Through the search engine it revealed 150 publications. However, duplications were 
removed and 33 publications were left. While undertaking in-depth screening of the 
left articles 15 were feasible. The PRISMA flow diagram illustrates the elimination of 
publications. Some of the publications did not provide sufficient information for the 
review but they aided in sufficient information to give context to the topic in matter. 
Table 1 shows a summary of the findings. 


3.1 Risk of Bias in the Selected Study 


The risk of bias was summarized in Review Manager (RevMan 5.4.1), and the feedback 
indicated that the overall risk of bias was moderately low. 

From the data collected, various information was present that was highly noticed. 
For example, persons known to have got infested with COVID-19 had an increased 
probability of being at risk of psychological impacts. According to Dhingra et al. (2021), 
students who had information of either one or more of their relatives was infected 
had higher chances of suffering from psychological compared to students who had no 
knowledge of anyone infected with the virus. Also, based on gender, women were at a 
greater peril of mental distress during the contagion as they were liable to depression 
and anxiety conditions compared to the male species of students as much as men were 
prone to get infected with the disease compared to the female. 

Universities would be well-served to discourse the conceptual wellbeing issues 
of their all-inclusive student body, given the enormous fraction of scholars con- 
signed to the high psychosomatic influence profile. Virtual group workout and con- 
sideration/mindfulness gatherings, liability partners and bodybuilding challenges, and 
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Fig. 1. PRISMA flow diagram of studies in the systematic review. 


telemedicine/therapy appointments are some of the activities that have endorsed mental 
well-being at the University of Connecticut, University of Kentucky, and Northeast- 
ern University (Browning et al. 2021). These group interactions may effectively reduce 
anxiety and the sense of remoteness that the students in this study felt. Scholars with 
clinical anxiety levels can gain from digital therapies. Since COVID-19 is likely to 
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Table. 1. Summary of the findings. 


S.no 


Author 


Alghamdi, A. 


Year 


2021 


Study design 


Systematic 
review 


Population 
study 


30 


Outcome 


The study found that high to 
moderate level of agreement 
with regard to the positive 
and negative impacts on their 
social and educational lives 
associated with Covid-19 
pandemic. Also, the impact 
of the pandemic on social 
aspects was higher than the 
educational aspect of 
students’ lives 


Abbasi, S. 
et al. 


2020 


Systematic 
review 


377 


From the sample size used 
137 male and 245 females 
took part in the study. 
Overall, 77% of the students 
had negative perceptions 
towards e-learning. 
Additionally, 76% of the 
students use mobile devices 
for their e-learning. The 
review proved that students 
do nor prefer e-learning over 
face-to-face teaching during 
lock down state of affairs 


Dhingra, S. 
et al. 


2021 


Meta-analysis 


260 


On the evaluation of 260 
responses, it was found 
majority of students were 
motivated to attend online 
classes but external factors 
like internet connectivity and 
home environment were 
among the greatest 
drawbacks 


Olum, R. et al. 


2021 


Systematic 
Review 


221 


In the study, the overall 
participant responded after 
removing some of the 
duplicates, 95% of the 
respondents possessed smart 
phones while 90% also had 
email address 
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Table. 1. (continued) 


S.no | Author 


5 Shahzah, A. 


Year 


2020 


Study design 


Theoretical 
review 


Population 
study 


107 


Outcome 


The study majors on 
comparison between male 
and female counterparts on 
e-learning portal usage 
among university students. 
The male species portrayed 
to have more experience and 
more conversant with the 
portal as compared to the 
female 


6 Muller, A. 
et al. 


2020 


Systematic 
Review 


14 


The study was conducted to 
get the experience and 
perspectives of university 
educators in Singapore. 
Educators had limited 
e-learning experience prior to 
Covid-19 sessions. 
Additionally, Hybrid or 
blended learning approaches 
were preferred, but support 
was necessary 


7 Yekefallah, L. 


et al. 


2020 


Systematic 
Review 


420 


The study was to find out the 
effectiveness of online 
learning during the 
pandemic. From a sample 
size of 420 students, 310 
were female and 2018 of the 
students were single. The 
review highlighted that 
satisfaction with e-learning 
was higher among female 
students and students with a 
history of attending online 
classes before Covid-19 


trigger persistent mental pain, colleges should emphasize aiding students in promoting 
proper mindsets rather than avoiding stress. Contemporary discoveries that intellectual 
and behavioral circumvention (i.e., evading contexts where disclosure is plausible and 
unpleasant beliefs about the contagion) was the utmost unswerving prognosticator of 
amplified anxiety and melancholy signs throughout the endemic. Stressful circumstances 
are getting addressed using cognitive reappraisal. Students are encouraged to focus on 
the “silver linings” and developing prospects in their educational experiences. 
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Universities can strive to evolve mechanisms that allow for safe social engagement 
among students. During their undergrad days, many students seek out social atten- 
tion. However, as this study’s findings indicated, students’ opportunities for mingling 
decreased significantly throughout COVID-19’s early stages. Our student participants 
consistently expressed dismay for missing “stepping out” and massive achievement 
events (e.g., graduation, last sporting occasion). In other research, students conversed 
online with close household members or colleagues no less than once a day to keep their 
mental health through the initial surf of the COVID-19 virus. 

E-learning. 

A cross-sectional expressive study gets piloted among students undertaking in varied 
fields of medical science university to try and find out what some of the students felt 
about the new learning technique facing out face-to-face learning that was already anorm. 
According to Abbasi et al. (2020), sampling gets piloted using the stratified arbitrary 
sampling method to capture all the faculties in the research and avoid biased results. 
Students were selected randomly and provided with an online questionnaire, partial 
responding to the questionnaire, and discontent with assistance were the measures for 
omission from the research. 

Data captured was composed using a three-part questionnaire that included demo- 
graphic and background information based on age, gender, year of study, and class 
performance prior and during the pandemic while also considering their experience 
in joining online classes before Covid-19. In total, 420 students got recruited for the 
research (100 percent partaking rate), with 310 of them being female (73.8 percent), 
208 having a single (49.5 percent), and 199 being sophomores (47.4 percent). The age 
of the participants was 21.14 (1.25) years on average (standard deviation). The mean 
(standard deviation) of the participants’ average point grade from the previous year was 
16.12 (1.48) out of a prospective 20 with regards to the table (Yekefallah et al. 2021) 
(Table 2). 


Table. 2. Demographic and background appearances and relationship with satisfaction with 
virtual learning in scholars. 


Qualitative variables Desirable Undesirable P-value x 
satisfaction satisfaction 
Number (percent) Number (percent) 
Gender Female 135 (78.5) 175 (70.6) 0.039 
Male 37 (21.5) 73 (29.4) 
Marital status Single 85 (49.4) 123 (49.6) 0.258 
Married 77 (44.8) 119 (48) 
Divorce and | 10 (5.8) 6 (2.4) 
death of a 
spouse 
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Table. 2. (continued) 


Qualitative variables Desirable Undesirable P-value x 
satisfaction satisfaction 
Number (percent) Number (percent) 

Academic years | Freshman 28 (16.3) 30 (12.1) 0.411 

Sophomore 85 (49.4) 114 (46) 

Third-year 41 (23.8) 51 (20.5) 

student 

Fourth-year | 18 (10.5) 53 (21.4) 

student 
A history of Yes 141 (81.2) 92 (37.1) 0.016 
attending online | No 31 (18.8) 156 (62.9) 
classes before 
Covid-19 
A suggestion for | Very high 31 (18) 53 (21.4) 0.339 
using e-learning High 58 (33.7) 60 (24.2) 
system 

Moderate 25 (14.6) 24 (9.7) 

Low 52 (30.2) 105 (42.3) 

Very low 6 (3.5) 6 (2.4) 
More desirable E-learning 7 (4.1) 12 (4.8) 0.253 
educational method 
method Traditional | 33 (19.2) 28 (11.3) 

teaching 

methods 

It makes no | 90 (52.3) 132 (53.2) 

difference 

Both 42 (24.4) 76 (30.7) 
Overall opinion | Very 18 (8.7) 36 (14.5) 0.181 
about e-learning | excellent 

excellent 49 (28.5) 62 (25) 

Moderate 40 (23.3) 57 (23) 

Weak 62 (36) 87 (35.1) 

Very weak 6 (3.5) 6 (2.4) 
Quantitative Mean + standard |Mean + standard | P-value + * 
variables deviation deviation 
Age 20.51 + 1.31 21.52 + 1.46 0.565 
Grade point average of previous | 16.28 + 1.58 16.09 + 1.79 0.414 


years 
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3.2 Data Analysis and Review 


From the data collected and analyzed, the outcomes signposted that there was a note- 
worthy relationship between gratification with online learning and variables of sex and 
previous experience of joining online classes prior to the Covid-19 outbreak. Addition- 
ally, the age bracket had a tremendous impact on the satisfaction of attending online 
classes. According to Yekefallah et al. (2020), they explain that with an increase in age, 
the level of gratification was similarly increasing, and there was no affiliation between 
male and female scholars, based on gender, in the aspect of e-learning. The variations 
in the arithmetical populace of the research and the use of various techniques to mea- 
sure the performance of education are the factors for this discrepancy. The optimistic 
worry of women for men and society’s culture and the boundless existence of women in 
societies might get cited as factors for this gender divide. Females consent to e-learning 
because their limitations do not preclude them from accessing many sciences. Further- 
more, the researchers found that 59 percent of them were displeased. Yekefallah et al. 
(2021) explain that Students’ satisfaction with the evaluation technique portrayed that 
self-assessment gave students the chance to assess and do a recap of what they have 
learned and their perceptive and metacognitive approaches. Also, students grasped their 
strong point and weaknesses. There was a deep and optimistic connection between the 
lecturers’ oral and non-verbal behavior and the prospect of using automated content to 
complete training. 

Under the scholars’ point of view in the digital learning course, the efficacy of the 
educational contented, the layout of a page to the desired level, the efficiency of teaching- 
learning activities, and assisting scholars to the desired level, and the springiness of 
online learning course have been moderate. However, e-learning can deliver prospects 
for scholars to acquire self-assessment methods courtesy of information technology, 
their level of relations, and feedback becomes minimal (Abbasi et al. 2020). From the 
research, it is quite important to mention that efforts have to be made to expand the value 
of virtual learning and the factor distressing it due to the pervasiveness of Covid-19. 
Distance learning may be held for the longest period ever since no one has any idea 
when the pandemic will come to a halt. Abbasi et al. (2020) argue that inattentiveness to 
these suitcases can lessen the eminence of edification and scholars’ level of acquaintance 
which would consequently, due to the unwelcome gratification with online learning, has 
generally reduced. 


3.3 Motivation 


Online courses are an impeccable option for eventful people with numerous burdens 
on their time. However, the tractability of distance learning brands it ideal for people 
held up with activities to have an adjusted learning schedule when they are off their 
work or during personal time. However, Sah Allam et al. (2020) explain that lack of 
self-discipline regarding the personal drive to learning influences readiness level owing 
to environmental disruption such as working obligation and massive home environment 
is a crucial challenge to online distance learning. Also, having friends’ interruption 
during online learning sessions by inviting coursemates to join the online conversation, 
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including chatting or even blathering, significantly impacts enthusiasm level and self- 
discipline. Hence, the student who lacks willpower and experiences lower stimulus can 
fall into the abrasion group of students that affect their school performance in distance 
learning. 

Sah Allam et al. (2020), talks of researchers who examine student online learning 
readiness, reported that computer internet literacy level has a more significant impact 
among students. Actually, so many students are ready to espouse online learning technol- 
ogy, although it has to counterpart with high competency in the literacy level. However, 
as much as so many students are into online distance learning, most of them lack the basic 
computer skills that hinder them from the e-learning platform, making the implemen- 
tation ineffective and thriving backward. An arduous study on student virtual learning 
attitudes reported higher computer internet levels correlated with higher online learning 
attitudes. Also, the enthusiasm to explore computer tools and have the capability to use 
internet applications could reduce the sign of technology stress among scholars, mainly 
in venturing into online distance learning avenues. Sah Allah et al. (2020). 

Having a self-drive towards distance learning enhances the verge of studying respon- 
sibility to become an independent learner, captivating as compared to traditional face-to- 
face learning and embedded sense of personal autonomy. Lassoued et al. (2020) explain 
that a scholar with a self-directed learning element possesses high self-disciplined, self- 
independent learning and high self-confidence to complete a duty in an online distance 
learning. However, the majority of students, especially during the pandemic period, 
may have difficulty presenting high performance on online distance activities during a 
pandemic outbreak since the means of study tends to seem peculiar to them and work 
solely, completing assessments with minimal or no contact with the lecturer, and unable 
to control their stress resulting to low triumph that affects their final results in studies. 

Motivation is the key for anyone determined to see the other end of the tunnel to excel 
in this form of learning since various milestones in the journey face it. Sah Allah et al. 
(2020) defines it as the engine of learning and go ahead to describe it as the fuel of human 
understanding. The numerous activities that generate learning byproducts demand time, 
money, and commitment. With all this, there is an urge to be some kind of motivation 
to make the process run effectively. With regard to the principle of abilities and human 
learning on the role of stimulus, for any student to avail themselves to a learning lesson, 
there has to be incentive or drive force raise. For a student to sit down for a class, be it 
online or physical, they have to be strongly motivated for higher academic achievements, 
and this is through establishing good study habits. Integration technology in learning 
does not positively enhance students’ level of motivation (Sah Allah et al. 2020). Online 
learning requires a student to have high internet connectivity while great effort and 
commitment for it to take place. 

For the success of e-learning, lecturers need to understand students’ motivation, 
while on the other side, students themselves need to adapt and shift from face-to-face 
learning into an online class. Sah Allah et al. (2020) states that the success or failure 
of online education is typically related to student motivation. Therefore, the lecturers 
have an obligation to keep in mind that motivation must get natured in students. This is 
by elaborating to students how the online environment may be exploited and embolden 
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interaction and teamwork among the students themselves to strengthen learning moti- 
vation. Also, creating study groups so that students do not study in isolation while still 
making friends by meeting other students as much as the social distance is upheld to 
prevent the virus from spread within the students and lecturers. Also, the professionals 
themselves need to be uplifted since they cannot work with students while they are not 
scythed up. According to research on distance education readiness El Refae et al. (2021), 
high education teachers show an optimistic attitude and enthusiasm to teach university 
scholars even with special needs like hearing despite the risk and challenges in learning 
equipment like pedagogy. Hence, with the pandemic being slated to stay with us until a 
protective vaccine is found, it is crucial to understand how the lecturers who are prime 
implementers of the learning accustomed to this switch and what encounters they face 
while adapting to the growth, and how well they are for the times ahead of us. 

As much as online learning came as a solution to curb the supper of the Covid-19 
virus among students and the whole world, it is faced with some challenges that have 
an impact on the delivery of what various university courses entail. So many courses 
undertaken at the university level get incorporated with practical classes in laboratories, 
especially ones that are quite difficult to undertake through e-learning. According to 
Plitnichenko (2020), practical classes require a face-to-face interaction of the student 
with the equipment involved while with the assistance of a technician and lecturer to 
facilitate the proper running of the experiments. It gives the students a much clear 
understanding and exposure of what they are learning about, getting a much valuable 
grasp of the class content. Hence, as much as containment measures have to be upheld, 
online learning cannot effectively facilitate practical classes for the student since virtual 
is unfit for practical courses. With that, it is up to the university as an institution to 
coordinate how practical classes will be conducted within the available laboratories and 
determine how much capacity each laboratory can hold, given the number of students 
in the university. 

Connectivity is the second most significant challenge facing e-learning since it is 
also dependent on it entirely. Plitnichenko (2020) argues that millions of people world- 
wide experience technical challenges due to the extreme usage rate of online learning 
systems, video streaming apps, and other learning tools due to overloading, which leads 
to poor video and audio quality. To be precise, internet connectivity is a crucial challenge 
worldwide and especially in third-world countries. Internet connectivity determines the 
quality of teach students access from these classes attended. With that, so many countries 
have not invested enough in the internet connectivity; hence this acts as a drawback to 
the growth of online learning. Plitnichenko (2020) explains that the critical issue is not 
just having a computer but getting a computer with internet connectivity since the speed 
of connection acts as a limiting factor. In order to address the issue, it is in high demand 
for various heads of state to uphold internet connection as one of the basic needs since it 
is becoming of great importance to our daily lives activities. Poon (2020) explains that 
Wi-Fi connectivity ought to be regarded the same as power and running water due to the 
monetary and profitable implications. It was felt as if people were so many years from 
that. 
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4 Conclusion 


Online is the future of where the world is heading to. Hence we have to move with the 
new trend that is approaching us. According to Swan (n.d.), the 21% century is rapidly 
becoming digitized, providing a valid reason for academics to be more aware of the 
specifics of their cohorts and learners and get ways to engage with and sustain isolated 
students. The reflection presented throughout this paper provides an overview of why 
e-learning should be embraced and given full support. With the pandemic being amongst 
us, preventing it from spreading is a better solution than curing, implying that e-learning 
is the resorted option left for academic institutions to run education with the condition 
at hand. 

The transition towards e-learning has been a difficult task not only to students but also 
to the facilitator across the world, and it has manifested itself in various avenues. Turner 
(2020) says that students describe the situation as having more workload to handle and 
being a significant reason why they struggle to stay on top of their school work during 
the transition. In addition, a large population of students is still inclined to face-to-face 
learning systems making online studies a hard-to-get approach to adapt to. However, 
e-learning is the only option left to handle many people at one time since the social 
distance is a crucial issue. 

Students have varied perceptions concerning e-learning, some being positive while 
others negative, which wildly contradict them, although they still have to attend classes 
and submit assignments. However, mobile is one of the most common devices among 
students for e-learning compared to laptops and tablets since it is cheaper and more 
easily portable than having a computer with you. Abbasi et al. (2020), explains that 
a study carried out on university students proved that 66% of the students use mobile 
devices for e-learning, which sound similar to that of a large percentage preferring 
having mobile device compared to having laptops for studies. This implies that online 
learning in universities could increase double if students gained an access to a reliable 
and convenient internet access since almost all have a smartphone that supports access 
to various websites and other learning platforms in the internet world. 

In a nutshell, education is the backbone of a nation hence regardless of what happens 
education has to continues as it was a much as possible. Also the Covid-19 pandemic has 
significantly impacted the education sector but learning institutions have the obligation 
to continue devising suitable ways to ensure learning activities continue without much 
deterrent. Additionally, based on the majority of articles we have reviewed it is quite 
clear education cannot proceed while using the traditional method and a change had to 
be initiated. E-learning has proven to be the newest learning process that we have to 
go with since medical practitioners have not clarified when Covid-19 will come to an 
end, and learning has to continue. Hence various learning institutions should invest in 
their digital platforms in collaboration with the Heads of state and provide convenient 
internet access to their students to facilitate proper university learning. Finally, learning 
institutions can implement some of the recommendations that we have highlighted as 
they would be helpful while revising some of their policies. 
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Abstract. Changes in digital storytelling are increasing the platforms for peo- 
ple in different professions such as business, family, healthcare, and research 
to communicate. Social movements through digital storytelling practices have 
increased over the past decade due to digital storytelling evolvement. Social activ- 
ities through social media platforms have impacted freedom of communication, 
and continuous changes in digital storytelling will create more space for peo- 
ple to raise views. Other practices affected by the evolvement of digital story- 
telling include marketing, healthcare such as mental health of people, teaching, 
and research. Generally, digital storytelling changes over the past ten years have 
improved communication skills, enhanced interpersonal relationships, teamwork, 
and interprofessional collaboration globally. 


Keywords: Digital storytelling - Education - Social media 


1 Introduction 


Digital storytelling (DST) has evolved in the presence oftechnological advancement over 
the past ten years, currently creating suitable and more inclusive stories. Consequently, 
digital stories are multimodal, multimedia presentations merging diverse digital aspects 
within tale structures. Instances of such media are video, social media aspects such as 
Tweets, images, audio, and cooperative facets such as digital maps. Therefore, digital 
storytelling is a digital media communications approach that lets individuals share their 
stories and experiences every day. Digital storytelling is utilized in various segments, 
including communication, research, marketing, psychology, and education. This study 
will emphasize three aspects of the digital version of storytelling: the studies, models, 
and practices. There are numerous studies on digital storytelling ranging in education, 
health, identity, and relationships. 

On the other hand, digital storytelling practices focus on how best they are conveyed 
to other people. Examples of such patterns of digital storytelling include health, iden- 
tity, strategic communication, and the creation of interrelationships. In such categories 
and many others, the best practices emphasize relevance and reaction, informing and 
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educating people. For instance, a healthy digital version of storytelling should provoke 
a response, and the person telling them must reach as many people as possible. 
Therefore, the digital understanding of storytelling practices is to connect emotion- 
ally with the audience to impact their actions. Digital storytelling depicts a challenging 
form of generating a connection between old style oral storytelling and the communica- 
tion form utilized in the modern era and information advancement. Therefore, models 
for digital storytelling describe the interventions used. The primary digital storytelling 
modes include videos, photovoice, and slide shows. Thus, the are also three implementa- 
tion models of digital storytelling. They constitute digital storytelling using slide shows, 
using photovoice, and using video techniques. Older people face several challenges in 
relation to psycho-physical health and may experience a range of psychological comor- 
bidities. Advances in multimodal, multimedia technology have made it possible to use 
digital storytelling (DST) as an intervention to improve health. A review by Stargatt et al. 
[1] focuses on the effects of DST on health in the elderly (cognitive and psychophysical 
well-being). The results of another review [2] showed that all stakeholders in the health 
system could be producers of digital stories with various objectives, such as community 
health, empathy promotion, attitude and behaviour change, clinical thinking, and skill 
improvement. A decisively interesting article, albeit one more concerned with the edu- 
cational aspect, is Yeh and Mitric [3]; it demonstrates ways to incorporate Instagram 
as a multimodal DST tool to (1) promote the motivation and engagement English lan- 
guage learners (ELLs) at the university level; (2) improve the oral and written skills of 
ELLs; and (3) help ELL’s stories and voices be heard. The advent of new media and 
new digital technologies have changed the classic storytelling methods. Starting in the 
90s, thanks to the studies of Joe Lambert and Diana Atchley [4], Digital Storytelling 
was born, that is a particular narrative technique that makes use of the use of digital 
tools. Scholar Meadows [5] defines DST as «short and personal multimedia stories, told 
from the heart», stating that «it is not just a tool, but a revolution». Most digital stories 
are collected in online repositories, such as the Berkeley Center for Digital Storytelling 
website, where the stories are divided by subject areas. Over time, the advent of social 
media has brought about a considerable change. In fact, social media have in a short 
time revolutionized the classic way of communicating, allowing a rapid and viral use of 
information. With this in mind, the way in which digital stories are used also changes. 
There is an increasing tendency to use social media platforms for sharing digital stories, 
as their circulation is much more viral and dynamic than traditional media and platforms. 


2 Background 


Over the past ten years, our universe and how we narrate stories has changed in a myr- 
iad way. For example, people have shifted from face-to-face physical levels to digital 
platforms to stimulate social movements via digital smartphones through social media 
accounts. First, in the past decade, there has been an evolvement of digital visual sto- 
rytelling. The rise of this form of storytelling has allowed numerous people to show 
stories more than before. Through such evolvement, it has added a storytelling dimen- 
sion that demands improved connection and creativity. Whereas images have regularly 
had a place in narration dating back to the cave paintings times, the development of 
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methods such as live scribing, explainer animations, and infographics change the way 
individuals “show,” listen to, and tell stories. Seeing concepts, projects, and dialogues 
brought to reality before people’s eyes is a game changer to how organizations or people 
intend to tell stories. Arising digital storytelling via visuals in the past decade built a 
space for tremendous inclusion of diverse learners’ types across cultural and language 
divides. 

Secondly, story narration has evolved into sharing people’s opinions and marketing 
trends through the digital brand story [6]. It is not enough to depend on the services 
and products’ uniqueness or dominance to market themselves in the current century. 
The global market necessitates and demands more than their dominance to flourish 
through the brand story. Through such stories, they engage people’s consciousness in 
ways that thus far has been difficult to achieve. The brand story enables one to share 
the products’ purpose with customers and prospects. However, digital storytelling for 
strategic communication purposes should be authentic enough to connect and resonate 
with audiences. For example, over the past ten years, we have witnessed the Airbnb 
company maximizing on the “belong anywhere” story, enabling it to stand out. The 
company has been carrying out branding from its website to campaigns attracting more 
customers through video storytelling. 

Thirdly, digital storytelling has evolved over the past decade to foster social move- 
ments, unlike with traditional story approaches. During this digital period, actions 
spark events across the whole human structure in moments. For instance, digital sto- 
rytelling through Tweets has witnessed social movements such as #BlackLivesMatter 
and #MeToo. They are successful because they focus on using multimodal affordances 
of digital storytelling across digital platforms that in turn facilitate social change. There- 
fore, the practices of these digital storytelling movements have been carried out through 
YouTube, Instagram, Twitter, and Reddit over the past decade. The evolution of such sto- 
ries, characterized by collective and united voices, have lead to new paradigms emerging. 
Consequently, new technologies have enabled the successful evolution of these digital 
stories since they have changed how people see, share, and listen to them. Digital tech- 
nology platforms have allowed individuals to compress time and geographical barriers, 
enabling awareness creation, engagements, and development of the digital stories locally 
and globally. 

Furthermore, digital storytelling over the past ten years has experienced more democ- 
ratization than before. People have had storytelling power in their hands since 2010, 
leading to democratic evolution. With such changes, we are now hearing and seeing 
various natratives and voices across diverse stands than ever before. If we contemplate 
the implausible power that digital stories possess to impact perception, build connec- 
tions, and make the universe change happen, storytelling democratization is considerable 
for all individuals. Consequently, within the past ten years, digital stories have allowed 
human beings to live the “mantra” (nothing about people, without people) and generate 
a more all encompassing and equitable universe. 


3 Studies on Digital Storytelling 


Over the past decade, there have been various studies on digital storytelling in multiple 
fields. Some studies have focused on how they have been utilized to support teaching and 
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learning in the education field. In this study, Robin [7] explains that digital storytelling 
in the past ten years has increased the number of students and educators worldwide 
through computerized short stories in the form of video, music, and audio. In the medical 
and health sector, some studies focus on digital storytelling as a technique in health 
research. According to Rieger et al., digital storytelling in the health sector is vital in 
developing a detailed comprehension of how researchers have utilized it as a technique 
aiming to refine its future iterations. Other studies base on the effectiveness of digital 
storytelling in education, revealing identity, family, relationships, and even human rights 
witnessed through social movements. These studies present various digital storytelling 
practices such as research, communication, educational learning, marketing, and cultural 
awareness and practices through tourism. 

Sports depression is a serious mental health problem among athletes. Ofoegbu et al. 
[8] conducted a study to determine the effect of a rational DST as an intervention for 
depression in a sample of adolescent-athletes with special educational needs. The par- 
ticipants consisted of 60 school-aged adolescent athletes with special educational needs 
(treatment group, n = 30, control group, n = 30) and moderate to severe depression. 
Hakanurmi et al. [9] studied the improvement of work quality and well-being in a business 
organisation through DST. A study by Kim et al. [25] in a clinical setting qualitatively 
analysed the metacommunication developed during the DST process for patients under- 
going haematopoietic cell transplantation (HCT). DST has been adopted for university 
courses, including sociology courses. In an experiment by Vaughn and Leon [11], the 
practice was useful for the following: (1) developing students’ sociological imagination 
and intersectional thinking by linking their own experiences to social theory, (2) devel- 
oping transferable technological skills, and (3) learning to apply sociological concepts 
outside the classroom. Rodriguez-Illera et al. [12] analysed the reasons that guide the 
publication of personal stories as well as the kind of filters, agents, and consequences 
articulated in these stories. For this purpose, the most qualitative part of a survey on 
guidelines for publishing adolescents on the Internet is summarised. Specifically, the 
results of a semi-structured interview with 12 adolescents aged 12—20 were shown. 

The responses showed that the respondents’ perspective on what constituted a digital 
story went beyond format and varied depending on the network it was published on as 
well as the type of content. Thus, the emphasis and motivation of the publication shifted 
from the digital, which was incorporated as something normal and non-differential, 
towards the personal, focusing the publications on the daily lives of young people. 
Liontas and Mannion [13] presented the main benefits of DST for teachers and students. 
The main objective of the review was to maximise the conditions of multimodal creation 
for optimal online and offline learning. It was concluded that students are effectively 
able to transform themselves from passive consumers of technology into active creators 
of digital compositions, of a voice and of a story. 

Computational thinking (CT) is a problem-solving skill that can motivate students to 
learn English. The “present, practice and produce” method has been applied as a method 
for presenting computational thinking in the English language learning class [14]. The 
results demonstrate the positive effect of the CT strategy on strengthening the problem- 
solving skills of students participating in DST and on increasing their motivation and 
performance in learning the English language. Nosrati and Detlor [15] explored how 
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cultural organisations in cities can use digital stories to enhance the public perception of 
a city. The aim of this study was to understand how end users are influenced by a city’s 
cultural DST information system and the benefits of using such a system. Meanwhile, 
Shiri et al. [16] examined the role of DST in the enhancement and preservation of oral 
traditions and cultural heritage. Schmier [17] highlighted how enabling primary school 
students to use digital tools to tell their stories has created new spaces for pupils to 
represent their identities. This has also educated them from the point of view of media 
and helped teachers plan practices of meaningful reading. 


4 Objective 


This review aimed to find and evaluate available studies of digital storytelling, practices, 
and its models. The objective will also constitute an in-depth evaluation of practices 
and models of digital storytelling. Consequently, the entire study will evaluate diverse 
digital storytelling aspects and their effectiveness in communicating the message to the 
audience. 


4.1 Overall Aim and Review Questions 


This review’s overall aim is to acknowledge and synthesize evidence on the studies, 
models, and practices of digital storytelling over the past decade. The questions that will 
direct this narrative review are as mentioned: a) What are the main foci of the studies 
and contexts of digital storytelling in diverse fields? b) What models are available for 
digital storytelling? c) What are the fundamental practices of using digital storytelling 
within various fields and in real life? 


5 Materials and Methods 


This narrative review will create evidence about the practices, models, and studies on 
digital storytelling in various sectors such health and education, maximizing on estab- 
lished review techniques used in solving health problems. It is a unique review since 
it focuses on the research technique instead of the intervention or phenomenon. Even 
though quantitative and qualitative studies and mixed methods will be included, only 
narrative information related to our study questions will be extracted. The PRISMA-P 
checklist [18] guided the progression of this review protocol. 


5.1 Eligibility Criteria 


Only research in English language and exploring the studies, practices, and models 
on digital storytelling was included. Including only English language articles is due to 
the cost of obtaining and translating other language articles. We considered published 
articles from 2010 to the present to align the evolving digital storytelling nature. Articles 
chosen had to have accurate abstracts and titles. Accurate abstracts and titles emphasized 
the studies and models touching on digital storytelling’s effectiveness and use in various 
fields such as education and health. The eligibility criteria were explained mainly to 
include the following attributes: 
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e Published in the English language Peer-reviewed 

e A standard item containing novel information 

e Focusing on digital storytelling in the education, health, and marketing sectors 
principally 

Published 2010 to present 

Focus on specified areas and data collection somewhat 


The articles chosen based on the above attributes were further reread and system- 
atized based on demonstrative sample quality. The evidence levels of the sample studies 
utilized in representation must have been determined based on a specific confidence 
grade in representative samples. The pieces had to have experienced digital storytelling 
effectiveness or impacts on that particular study. 


5.2 Intervention and Outcomes 


This narrative review will include primary research studies that utilize digital storytelling 
as a technique at any point in the study process (data collection, analysis, recruitment, 
or knowledge translation). On the other hand, outcomes will be extracted related to how 
digital storytelling impacts the individuals and their activities over the past ten years. 
Such actions can improve health, education, and business marketing. 


5.3 Search Strategy and Information Sources 


Informed by an initial literature search for “digital storytelling” and its deviations, var- 
ious primary search databases were used. They include Google Scholar, Art Full Text, 
PsycINFO, MEDLINE, EMBASE, PubMed, and Cochrane Library. 

Regarding the inclusion and exclusion criteria used here, the articles were selected 
from peer-reviewed English journals that aimed to describe or evaluate the dimensions 
and variables expressed with respect to the research topic mentioned above (screening). 
Articles from publications that did not deal with the topic in relation to the fields of 
education/psychology, communication, marketing, as well as those for which the com- 
plete text (relevance) was not found, were excluded. A qualitative synthesis of the most 
relevant information was also conducted with comparisons done between the various 
publications. 

The search on these databases reflected the range of likely terminology to capture 
studies appropriate to digital storytelling evolvement in the past ten years. Search tools 
and approaches particular to each database is used, including keywords truncation where 
suitable. It includes using subject headings and thesaurus terms and compounding search 
and terms strings with the eligible Boolean operators. For the comprehensiveness, papers 
that employed mixed methods, quantitative, and qualitative techniques were considered. 


5.4 Data Extraction 


The selection procedure started with reading the abstracts and titles of the considered 
articles. After identifying and excluding the abstracts that rarely seem satisfactory, the 
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remaining articles were fully explored. Articles and papers identified as appropriate for 
the narrative review process were inspected, and considerable information on the studies, 
models, and practices was extracted. Data extracted will comprise purpose, setting, 
benefits, limitations, procedures, and ethical standards of digital storytelling practices, 
studies, and models. Other collected information was reported from the commencement 
of the review where representatives of the chosen populations faced threats or biasness 
in looking for the studies. Limitations to the strategies used and research efficiency to 
allow their reduction was similarly taken as considerable information. The data collection 
setting observed the predetermined questions technique defined by the review research 
questions. The extracted information from selected studies will include: 


5.4.1 Title, article/journal title, authors, publication date 

5.4.2 Digital storytelling description 

5.4.3 Role of digital storytelling 

5.4.4 Findings or results of the study relevant to digital storytelling 


5.5 Data Synthesis 


A narrative review is a type of publication that describes and discusses the state of the 
science of a specific topic or theme from a theoretical and contextual point of view. This 
type of review article does not list the types of databases and methodological approaches 
used to conduct the review or the evaluation criteria used to determine the inclusion of 
retrieved articles during database searches. 

A narrative review consists of the critical analysis of the literature published in books 
and electronic or paper journals. Narrative reviews have an important role in continuing 
education because they provide readers with up-to-date knowledge about a specific topic 
or theme. This type of review does not describe the methodological approach in a way that 
would permit reproduction of analyses based on the data or answer specific quantitative 
research questions [19]. Care was taken to prevent falsely presenting the results and 
avoiding the unsuitable focus on the results of a specifically selected publication for the 
research. Consequently, the data synthesis approach resided on the following procedures 
and comprised the following: 


5.5.1 Review of the effectiveness of digital storytelling as it evolved over the past ten 
years. 

5.5.2 Description of the models and practices from the digital storytelling point of view. 
5.5.3 Supposed weaknesses, contributions, and strengths of each digital storytelling pub- 
lication included. It was carried out depending on how they analyzed digital storytelling 
in each field they covered. 


Generally, the extracted narrative data will be presented in an evidence table ordered 
by study design and research procedure/stage integrating the digital stories. Provided 
that this narrative review will evaluate how digital storytelling has been employed across 
different fields, we will not emphasize synthesizing the study findings of the included 
research. Still, we will assess digital storytelling’s impact in various areas such as medi- 
cal, educational, and marketing or between individuals. Additionally, the synthesis will 
evaluate digital storytelling models and practices. 
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6 Results 


6.1 Selected Studies 


The databases produced 300 references. In this case, 50 were published in PubMed, 50 
in Medline, and the remaining were in EMBASE, Google Scholar, and Cochrane. 

The process we undertook for including studies in the systematic review included 
over 300,000 results. After eliminating duplicates and articles in languages other than 
English, the search identified 301 studies consistent with the search parameters. After 
excluding publications that did not fit with the selection criteria (n = 245) and those that 
had to be excluded because their content or the ways that the researchers applied the 
concept of digital storytelling the areas of application of digital storytelling fall outside 
the parameters set for the research sample (n = 31), 25 remained that met the inclusion 
criteria. The studies were integrated with a supplementary keyword search on the latest 
trends in the literature on the topics addressed in this study. 

After evaluating the articles based on the eligibility as mentioned above criteria, 25 
studies were selected. The high number of similar research articles depicted that there 
has been a more concern with studies on digital storytelling over the past decade. There 
are diverse research topics about digital storytelling, its models and practices in various 
fields from these articles (Table 1). 


Table 1. Assessed selected articles. 


Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
Ellison and Wang, Digital The study Family literacy Narrative 
2018) [43] Storytelling focuses on digital | practices model of 
Workshops in storytelling Digital literacies digital 
university between African storytelling 
computer labs American son 
and participants’ | and mother 
home 
(De Vecchi et al. Digital databases | The study Psychology Interactive 
2016) [44] for emphasizes how | (mental health) model of 
peer-reviewed digital digital 
literature storytelling is storytelling 
utilized in 
supporting 
stakeholders of 
mental health 


(continued) 
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Table 1. (continued) 


Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
(Ribeiro 2016) [45] | The Polytechnic | Education. The | Teaching with a Narrative 
School in Porto, | study primary focus on — and 
Portugal in emphasizes communication interactive 
Business identifying the practices models 
Communication | practices and 
Course how students feel 
about digital 
storytelling in 
communication 
(Pera and Viglia Airbnb’s social | Marketing with a | Digital stories in Interactive 
2016) [46] platforms primary focus in | aiding consumer 
online experience 
peer-to-peer relationship 
communities 
(Rieger et al. 2018) | Critical Healthcare Digital Narrative 
[47] Databases and research storytelling’s model 
grey literature significance in an 
in-depth 
understanding of 
health researches 
(Beierwaltes et al. Nursing Education in the | Provision of an Interactive 
2020) [48] healthcare healthcare educational model in 
databases setting intervention to healthcare 
nurses in settings 
healthcare practice 
and focusing on 
family practices 
(Kasemsarn and Thailand Tourism and Digital storytelling | Interactive 
Nickpour 2016) [49] marketing in increasing model 
tourists’ motivation 
(Blue Bird Jernigan | California, Research Investigating food | Narrative 


et al. 2011) [50] 


United States 


security and using 
digital storytelling 
as the approach in 
focus groups 


(continued) 
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Table 1. (continued) 


Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
(Urstad et al. 2018) | Norway: two Healthcare Exploring the Narrative 
[51] diverse nursing nursing students’ model 
education experiences digital 
institutions in storytelling as a 
the country reflection tool 
during clinical 
placement studies 
(Clisbee et al. 2019) | Continuing Healthcare to Adapting Interactive 
[52] education promote approaching in digital 
workshop via continuous creating digital storytelling 
Academic Team | education to stories to minimize | model 
and empower nurses | logistics concerns 
Translational and overcome 
Partners digital storytelling 
implementation 
(Alexander 2017) The creation of | The new Digital | New platforms and | Narrative 
[53] Digital Storytelling new media for models 
Storytelling telling stories 
through new 
media 
(Canella 2017) [54] | Denver, Collaborative Social networks Narrative 
Colorado media-making and community and 
projects assist in | screenings interactive 
building local models 
grassroots 
campaigns of 
racial and 
economic justice 
(Cersosimo 2020) Digital Digital Literacy practices | Narrative 
[55] Storytelling in storytelling as Digital literacies model 
social sciences | the telling of 


stories through 
the network of 
digital media 


(continued) 
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Table 1. (continued) 
Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
(Chan and Holosko | Hong Kong, The use of social | Online searches, Interactive 
2017) [56] Japan media in youth initial encounters | model 
outreach before direct verbal 
engagement in dialogue, 
Hong Kong ice-breaking via 
multi-modal 
communication, 
and snowballing 
(Chan and Sage Digital A narrative Digital Storytelling | Narrative 
2021) [57] Storytelling in review of digital | as a kind of model 
Social Work storytelling for | narrative practice 
social work 
practice 
(Pettengill 2020) [58] | A collaboration | Social Media and | Social media asa | Narrative 
between Digital tool for accessing | and 
Michigan-based, | Storytelling for | and connecting interactive 
online water Social Good with already models 
journalism established 
organization and communities 
Ball State 
University in 
Muncie in 
Indiana 
(Pireddu 2019) [59] | Britain Creative works Storytelling Interactive 
as an ongoing combines oral and 
‘emergent’ narrative (mode) in | narrative 
process at the the form of a script | models 
intersection (genre) but uses a 
between porous | variety of media to 
media create meaning 
boundaries 
(Podara et al. 2021) | Digital Storytelling in The Greek Interactive 
[60] storytelling in the context of interactive model 
cultural heritage: | interactive documentary NEW 
Audience documentary LIFE as a case 
engagement in study 


the interactive 
documentary 
new life 


(continued) 
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Table 1. (continued) 


Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
(Smed et al. 2021) Finland Interactive Platform, designer 
[41] storytelling and interactor 
(Moradi, Chen 2019) | Cina A systematic The DST is a tool | Narrative 
[61] review for the for mediating model 
use and language education 
implementation 
of digital 
storytelling in 
education, 
specifically 
language 
education 
(Rolbiecki et al., Colombia Digital Qualitative data Narrative 
2021) [26] Storytelling as a | revealed that for model 
bereavement some, participation 
intervention in Digital 
Storytelling 
facilitated growth 
and 
meaning-making 
(Ofoegbu et al. 2020) | Nigeria Effect of digital | The digital Interactive 
[28] storytelling storytelling model 
intervention on | intervention based 
burnout thoughts | on REBT 
of adolescent significantly 
reduced burnout 
thoughts among 
disabled 
adolescent-athletes 
(Rodriguez-Almagro | Spain Digital They believe that | Narrative 
et al. 2021) [62] Storytelling as a | this teaching model 
Teaching technique is more 
Method for stimulating and 
nursing students | more enjoyable 
than the 
conventional 
system, giving 
them more 
motivation to study 
the subject 


(continued) 
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Table 1. (continued) 


Author Place of Study | The field in Outcomes 
which the study | practices Models 
focuses 
(O’Byrne et al. 2018) | United States Digital Students develop | Interactive 
[63] storytelling to enhanced model 
support students’ | communication 
learning by skills by learning 
encouraging to organize their 


them to organize |ideas, ask 
and express their | questions, express 


ideas and opinions 
knowledge 
(Kim et al. 2020) United States Trial of a Digital | A digital Narrative 
[42] Storytelling storytelling model 
Intervention for | intervention seems 
Smoking to be a valuable 
Cessation in strategy to enhance 
Women Living _ | the effect of 
With HIV conventional 
tobacco 
dependence 


treatment for 
women living with 
HIV 


6.2 Methodological Quality Assessment of Included Reviews 


The two reviewers assisting with this review independently executed this assessment. 
However, these two reviews conducted pilot testing to assess the three review questions 
before the actual evaluation. The outcome showed suitable and reliable review questions 
reliability (depicting 85% agreement). They also performed the second pilot test on 
another review to evaluate if the understanding of reliability had improved. The second 
review pilot test showed 100% agreement depicting an improvement in reliability. Dis- 
agreements in the evaluations were managed in a consensus dialogue after comparing 
differences between the two assessors on digital stories. This narrative review has the 
following characteristics [20]: 


e research method: not predefined based on a protocol, which implies subjective 
selection bias 

e inclusion of studies: undertaken based on authors’ intuition and research experience 

e data extraction: non-protocol-based and providing a simple description of study results 

e data quality: partially objective classification using anecdotal resources 
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6.3 Certainty of Evidence Assessment 


To assess certainty in the general evidence body, we utilized the GRADE approach. 
Guyatt et al. [21] highlight that certainty of the evidence is how certain it is that the 
actual intervention effect lies within the selected range or on one side of a quantified 
threshold. In this review, the 95% of confidence interval was used the specified range, 
or the instituted MID (minimal importance difference) as the selected threshold. When 
available, GRADE and risk of bias assessments were utilized made by the included 
reviews’ authors. When not available, GRADE was applied and assessed the potential 
limitations due to the bias risk, imprecision, inconsistency, and indirectness grounded 
the original studies. 


6.4 Individual Studies Results 


Individuals’ articles were categorized as either representing an individual, group, or 
organizational-focused approach. Contemplating the individuals’ category, the primary 
concern was the qualification and improvement of personal internal resources. Such 
could be carried out through the individual’s dynamic behavior. Generally, the articles 
present a customized approach where the instruments perform considerably to empower 
individuals in digital storytelling via expanding their knowledge and skills. Concern- 
ing group approaches, digital storytelling practices and models promoted interprofes- 
sional collectiveness, interpersonal relations, teamwork, and good communication. The 
research articles advocate that digital storytelling has been the core of transition from 
analogue to digital stories over the past ten years regarding the organization mechanism. 
Improvement of the instruments and models used during digital storytelling can boost its 
practices in various fields. All the studies examined present diverse approach forms of 
practicing digital storytelling and what is perceived as the ultimate advantage: improved 
quality of digital storytelling practices (Table 2). 


Table 2. Types of digital storytelling practices per category 


Digital storytelling practices Category type % 


Perfection of communication Individual 5% 
Stress management through web-based moves Meditation through 
digital platforms 


Communication Interpersonal relationships Group 80% 
Interprofessional collaboration Teamwork 


Improvement of work environment Organizational | 15% 
Incentives for healthy leaving and psychological improvement 


7 Discussion 


This narrative review concentrated on the frameworks, practices, models, and purposes 
for which digital storytelling is utilized in various platforms, circumstances, and fields. 
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Similarly, it outlines the impact of digital storytelling in such outlined areas and people’s 
behavior on its usage. Given the popularity of social media platforms, for instance, Insta- 
gram (informal digital stories), Twitter (social movement digital stories), and Facebook, 
we speculate that diverse fields will continue to embrace and utilize digital storytelling. 
Such outcomes are because individuals, organizations, and even firms use digital story- 
telling for communication, marketing, education, and the health profession. Currently, 
this review depicts that most digital storytelling studies over the past ten years are hap- 
pening in educational, health care, and marketing settings. These findings emphasize 
that education, health, and marketing fields often utilize digital platforms, constructive 
environments, and collaborative learning in different practices. In a medical or health 
setting, there is a developing interest in arts-based education which entails digital sto- 
rytelling. Such pedagogy utilizes art as a channel to back knowledge progression in 
subjects other than art. People can create art such as digital stories or respond to that 
of others to learn specific matters. Research reveals that the digital stories pedagogy 
can foster cultural awareness (for example, through tourism), community partnerships, 
social collaboration, self-awareness, observational skills in nursing, and whole personal 
progression. Therefore, as interest in arts-based teaching intensifies, were more confi- 
dent that digital storytelling will become more predominant in various professions in 
the coming years. A significant question raised by this review’s findings is: what counts 
or constitutes digital storytelling? As defined in the introduction, digital storytelling 
necessitates that it combines first-person and stand-alone narratives with multimedia. 
Therefore, from this review, digital storytelling constitutes or counts digital platforms 
and is used for diverse practices. From this review, communication, health, marketing, 
and family practices through digital storytelling have been more critical in the past ten 
years. Similarly, interactive and narrative models have been used in such practices over 
the last ten years. 


8 Conclusion 


This narrative review assessed 25 empirical studies. All studies focused on digital sto- 
rytelling, as well as different fields of tourism, then marketing, communication, health, 
such as psychology and education. Therefore, the assessed studies reported on the uti- 
lization of digital storytelling for varied, diverse aims, practices and covered an extensive 
range of topics. The authors emphasized the following issues: 


Family interactions and learning from generating their digital stories 

Health practices and professionals’ learning from the variation of digital stories 
Digital storytelling variations and impact on tourism and marketing 
Communication practices in digital storytelling over the past decade 

Digital storytelling as a research method in the health profession 

Digital storytelling effects in supporting learning and education 


Overall, creating digital stories in different fields enhances the understanding, pro- 
motes cultural activities, research, and gives individuals the platforms to express them- 
selves. There is a growing number of publications establishing DST as an effective 
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strategy for improving hypertension outcomes. DST reduced unscheduled meetings, 
and the programme was deemed feasible [22]. The study by Enverga et al. [23] aimed to 
explore the lived experiences of school-age students while digitally narrating the story 
of a boy and his undesirable food choices. The participants exhibited varying degrees of 
connection and understanding with the digital story and no doubt encountered a learn- 
ing experience through the digital story presented. DST supports student learning and 
enables teachers to adopt better and innovative teaching methods. Fiction is a proven 
and popular pedagogy; however, DST is relatively recent and still rarely used in early 
childhood education. Using a case study of “fiction — art — science club” in Jakarta, 
Indonesia, Rahiem [24] explored how and why DST is used in early childhood educa- 
tion. It was found that DST is engaging and communicative. Kim and Li [10] examined 
how DST has facilitated middle school students’ reflection and learning. It showed how 
to incorporate DST into a school curriculum to develop expressive resources and identity 
by improving motivation, creativity, and connection with others. Rolbiecki et al. [26] 
studied bereaved family members. The data revealed that participation in DST induced 
various effects: (1) writing and verbalising the script helped participants organise their 
thoughts and emotions about the loss; (2) having the space to share with a collective 
group enhanced their confidence in their ability to discuss their feelings with others; and 
(3) the final product served to process and distance themselves from the event. Hsieh 
[27] probed the positive and negative effects on students and their perceptions as a result 
of various presentation modes (robot-assisted or PowerPoint-assisted) using DST. This 
study involved 52 eighth-grade students. Despite the positive emotions experienced in 
both modes of presentation, the results of negative emotions were more mixed, as the 
students in both groups highlighted the feeling of fear during preparation and presen- 
tation. In addition, some trials [28] used DST intervention based on rational emotional 
behavioral therapy, which shows great potential in addressing burnout among adolescent 
athletes with disabilities. Furthermore, DST is an effective method for reducing anxiety 
in patients who are candidates for open heart surgery [29]. Patients with post-traumatic 
stress disorder (PTSD) can recall their memories in a digital diary and recreate them in a 
3D WorldBuilder. During therapy, a virtual agent is present to inform and guide patients 
through the sessions, using a question form to recall traumatic memories [30]. This 
system has the potential to be a valuable addition to the spectrum of PTSD treatments, 
offering a new type of virtual agent-assisted home therapy. VoicingHan [31] is a project 
that uses digital avatars that promote the reworking of the meaning of one’s life for dying 
patients. VoicingHan enrolled 12 patients receiving outpatient palliative care at the VCU 
Massey Cancer Centre. The narrative performances were recorded via the avatar video 
format to build the life re-meaning process. One study developed a marketing simulation 
through DST to facilitate the learning of marketing strategies; this is a useful tool for 
entrepreneurs that includes communication techniques, emotional involvement, and user 
persuasion [32]. The aim of the study by Fischbach and Guerrero [33] was to explore 
how the DST of a brand’s story creates a transformative learning experience for students. 
The tool is therefore digital brand storytelling, a new and contemporary application area 
of this DST model. 
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The applications of DST include its link with creativity. For instance, using creative 
DST to help crowdsourcing companies [34] brings out the persuasive nature of the prac- 
tice. In the context of learning, the goal was to evaluate the development of students’ 
creativity through the practice of science-centred DST [35]. The students turned their 
personal values into stories; hence, this creative procedure was influenced by social, cul- 
tural, and ethnographic characteristics. DSTs conveyed on social media through hashtags 
also lead to some political discussions. This is the case for the narratives on the theme 
of LGBTQ rights and freedom of speech to challenge political censorship [36]. Social 
participation and democratic commitment have often accompanied the application of 
DST. In fact, DST is a democratic tool for mediating knowledge and a mode of scientific 
diplomacy [37]. Dissemination has produced science-citizen interfaces highlighting cit- 
izen engagement in critical scientific debates. Within this sector, there exists a niche of 
studies conducted on the relationship between DST and social movements [38]. Analysis 
in the context of applying social participation and democracy through qualitative data 
indicated the power of the feminist DST for opportunities, access, validation, and heal- 
ing with respect to HIV disease, promoting social equity in sexual health, or advocating 
for abortion rights in Ireland [39] or as a bereavement intervention in the family [26]. 
Special needs are also experimenting with this practice as an inclusive laboratory, such 
as in the project that used an innovative DST to explore the experiences and perspectives 
of five four-year-old autistic children and their families. As the children prepared to 
make the transition from inclusive day-care to primary school [40]. The authors tried to 
answer two related questions: (1) How do we listen to children who “have no words”? 
(2) What do we learn from them when we do so? 

Rahiem [24] suggested that teachers’ ability to use DST should be improved. 
Information and communication technology (ICT) devices in schools should be more 
widespread, as governments need to modernize school equipment, which is rapidly 
becoming obsolete. Similarly, the curriculum should be adapted to meet technological 
developments and provide opportunities for children to learn how to make good use of 
technology. 

New digital media and new technologies have revolutionized the way of storytelling; 
starting from the 90s, we start talking about “Interactive Storytelling” on the basis of some 
studies done on the presentation of stories on two-dimensional screens [41]. “Interactive 
Storytelling” means a particular storytelling technique in which the reader is an integral 
part of the story itself. 

Unlike conventional storytelling, where the author of the book has full control over 
what happens in the story, in Interactive Storytelling users or readers can transform the 
story in real time, based on a series of interactions with the content and of choices made. 
The plot, in fact, is not static but varies on the basis of the choices made by the reader. 
When structuring an “Interactive Narration” it is necessary to take into account four 
fundamental elements: 


The platform - in which to build the story; 

The designer - who structures the story; 

The interactor - who modifies the story on the basis of his choices; 

The Storyworld - which includes all the characters, props, scenes and events 
organized by the designer for the interactor. 


aN 
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The designer structures the skeleton of the story on an ad hoc platform (providing 
the characters, external events and props), which is however experienced and influenced 
by the interactor. As the interactor makes his choices, various instances of the story are 
created. Since the interactor is the one who mainly experiences the story as it unfolds, 
he generally plays the role of the protagonist. 

Furthermore, the choices made by the interactor can be more or less significant, 
changing the story significantly or imperceptibly; it follows that the degree of interaction 
with history also varies. 

The degree of interactivity of the story can be influenced by three factors: 


e Speed: the faster the turnaround, the better the chances of interaction. This means that 
stakeholders can react faster and see the result of their actions faster. 

e Depth: it concerns the human similarity of the interaction. It takes into account not 
only simple cognitive modalities, such as hand-eye coordination or spatial reasoning, 
but also social reasoning, which is fundamental in these types of history. 

e The choice: which concerns both the functional meaning of the interactor’s choices, 
and the perceived completeness, or the number of existing choices compared to those 
that the interactor can imagine. 


9 Limitations 


Foremost, while the review attempted to identify relevant search terms, provided the 
searching complexity within digital storytelling topic in the past decade, the study might 
have unintentionally omitted keywords or synonyms utilized for digital storytelling. 
Therefore, the review could have missed empirical studies on this theme. Digital sto- 
rytelling dates back to the 1970s. The ten-year criteria of this study might exclude sig- 
nificant studies regarding the practice. Therefore, it could have missed older or recently 
published studies on this theme. Thirdly, most studies assessed have focused on digital 
storytelling practices without including the cause of changes in digital storytelling over 
the past ten years. Lastly, there is no recognized standard for gauging the changes in dig- 
ital storytelling in various fields. Instead, the studies assessed utilize practices and how 
they impact people and activities in their particular fields as the standard for measuring 
digital storytelling changes. 


9.1 Implications for Digital Storytelling Practices 


Individuals can benefit as much as organisations by incorporating digital storytelling 
practices in their daily activities. Communication, marketing, psychology, health, and 
education are dynamic fields and require people to engage on deep levels to be effec- 
tive. Digital stories offer the potential to meet these expectations. The significance 
of digital storytelling is that it creates a collaborative framework, promotes effec- 
tive communication, and enhances better interpersonal relationships among diverse 
professions. 

The application of DST is no longer restricted to the educational field. The review 
shows that DST is adaptable to various forms of intervention to promote personal 
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well-being [43, 44]. However, there is a lack of analyses that demonstrate this using 
quantitative methods. 

This review highlights the necessity for additional high-quality research on the impact 
and usage of digital storytelling in educational, research, communication, marketing, and 
psychology fields. 


9.2 Implications of the Review 


It is evident that digital storytelling practices and models are changing. However, these 
practices cannot be solely relied on as the standards of measuring the changes. It should 
include all significant changes over the years and their impact on individuals. However, 
while it is impossible to adopt and establish a standard measurement approach for digital 
story changes, digital history workshops should promote and publish not only the changes 
but also the implications on people, to improve familiarity with variations. 
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Abstract. The Problematic Use (PU) of Social Networking Sites (SNS) is a diag- 
nostic and preventive as well as educational challenge. Problematic Use of new 
media is currently a phenomenon discussed by psychologists, sociologists, and 
media educators in the field of diagnosing the scale of the phenomenon, as well 
as protective factors and risks related to this phenomenon. This text is part of 
the debate on the scale of SNS among young people, as well as on the role of 
parents in reducing this phenomenon. The text also juxtaposes issues related to 
the prediction of PU SNS and risk behaviours in the offline space. The study 
employed a triangulation of quantitative tools in the form of: frequency of SNS 
use, parental control online, and scale of psychoactive substance use. Based on 
the data collected among a group of adolescents in the Czech Republic (N = 
531 individuals aged 13-19 years, study year 2018/2019), it was noted that: 1) 
Almost 75% of adolescents systematically use SNS before bedtime; 2) Every fifth 
adolescent consumes dinner daily or almost daily accompanied by SNS; 3) Less 
than a third of respondents use SNS almost continuously; 4) SNS UI indicators 
are mutually related; however, the relationship is not always strong; 5) Girls have 
a slightly higher level of PU SNS than boys; 6) Systematic alcohol consumption 
is a predictor of PU SNS; 7) Parental restriction of Internet use time leads to a 
reduction in PU SNS among adolescents. 


Keywords: Problematic internet use - Social networks - Adolescents - Czech 
Republic - Offline risk behavior - Parental control - Parents 


1 Introduction 


The current stage of development of the information society is characterized by an 
increasing intensity in the use of digital services [1, 2] and a simultaneous increase in 
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awareness related to the positive and negative impact of new media on individuals and 
social groups [3]. Paradoxically, the development of new technologies, which by defini- 
tion was supposed to contribute to the improvement of the quality of life of people using 
information and communication technology (ICT), has revealed itself to be something 
of a trap at the same time. It transpires that selected digital services have the potential 
to negatively affect selected social groups. One example of such a condition is Prob- 
lematic Use of the Internet (PUI). PUI consists of intensive, unsupervised use of digital 
services, in particular Social Networking Sites (SNS), instant messaging, online shops, 
online games, gambling sites, and pornography sites; as well as proving a behavioural 
challenge, it has also become something of a diagnostic and preventive challenge as well 
[3]. PUI is an umbrella term that covers various forms of uncontrolled SNS use. The 
groups at particular risk for out-of-control use of SNS and online games are primarily 
adolescents [4, 5]. Considering that adolescents are a group at particular risk there is 
a need to undertake in-depth research on the scale of the phenomenon in this group 
with a particular focus on countries where research on PUI is rarely conducted on large 
research samples [6, 7]. The present text fills the gap in relation to one type of PUI, that 
of the style of SNS use among Czech adolescents. The text is also an attempt to discuss 
the concept of excessive SNS use and to link this concept to the broader framework 
of Problematic Internet Use, which is sometimes erroneously identified with Internet 
addiction. 


2 Theoretical Framework 


Problematic Internet Use is a theoretical concept that emerged as an attempt to describe 
the colloquial term Internet addiction. Due to the unclear diagnostic criteria associ- 
ated with digital addiction, the concept is controversial among both experts in media 
psychology and pedagogy and young people who use new technologies intensively 
[8-10]. 

The concept of PUI is a safer term, not only because it does not evoke stereotypes, but 
above all because it offers space for a discussion on diagnostic criteria and effective ways 
of counteracting the excessive and uncontrolled spending of time on the Internet. The 
issue of combatting PUI is also an element in strengthening digital security, i.e. raising 
awareness and shaping habits related to controlled use of new media while maintaining a 
high level of reflection relating to how ICTs transform users’ habits [11]. Studies devoted 
to PUI, or ICT use more broadly, contain many elements. On the one hand, one can find 
studies devoted to the constructive use of ICTs, where new media satisfy different needs 
of individuals [12, 13], while on the other hand, there are indications that ICTs make that 
relationship shallower, triggering conflicts that result from the time spent using ICTs in 
the immediate family environment [14]. In general, since ICTs are characterized by a 
high level of duality, the analysis of selected PUI factors requires particular care not to 
create additional stereotypes about the functioning of selected age groups in cyberspace. 

Problematic use in this text will be understood as a situation related to the misuse of 
online services that lead to conflicts with the environment and have a negative impact 
on the quality of life of individuals, and that occur in inappropriate situations. Among 
the types of PUI there can be distinguished the Problematic Use of social networks (PU 
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SNS). Due to the popularity and ubiquity of SNS, as well as the place that SNS occupy 
in the lives of young people, it seems reasonable to distinguish this type of PUI. Among 
the diagnostic criteria of problematic SNS use is, first of all: the place of SNS use, the 
time of SNS use, the type of activities undertaken in SNS, the type of needs that SNS 
satisfy, and the type of negative consequences that they cause in the individual and from 
a macro-social perspective. Due to the multiplicity of indicators, this study uses a simple 
SNS PU scale [15], which takes into account only the key parts of daily life currently 
affected by active SNS use. 

In this text, PU SNS will be defined as the use of different types of Social Networking 
Sites in an uncontrolled manner. SNS generate many individual and social problems. 
Individual problems include: loss of control of one’s own behaviour, reduced mobility, 
reduced physical fitness, reduced activity in cyberspace, reduction of one’s own life 
to virtual communication spaces, building one’s own identity on the basis of SNS, 
and deteriorating school grades. Social problems include conflicts in the immediate 
environment due to the intensive use of SNS, and a lack of social interaction (especially 
peer interaction) outside SNS (Fig. 1). 


Problems with self-controlled social network 
sites use 


Problematic use 
še of SNS 


fy] Social networks as Instagram 


a source of Facebook 
communication and Tik-Tok 


fin} (f) entertainment Snapchat 


| Ease of access 
| Entertainment centre 
| Place of communication 
| Space for self-creation 
| One of the basic e-services for young people 


Fig. 1. Characteristics of problematic SNS use 


Diagram number 1 presents the typical features associated with SNS. The popularity 
of SNS is due to several features of the development of the information society. First, 
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access to this type of service is now provided by easily accessible devices (e.g., smart- 
phones) with unlimited Internet access. SNS are characterized by an attractive visual 
interface, enabling quick and seamless sharing of information and networking with other 
users. SNS provide access not only to attractive pieces of information created by other 
users and commercial entities, but are also an entertainment center. SNSs are a space for 
the ready creation of one’s own image, through posting photos, descriptions, comments, 
and the interactions that occur with other users [16]. SNSs have become fundamental 
among the e-services used by adolescents that do not require advanced digital compe- 
tence. SNS developers stimulate user activity through a system of notifications about 
other users’ activities and information profiling. It is also increasingly noticeable that 
SNS use algorithms to target selected age groups and the type of content viewed and 
posted by users. Due to all the mentioned features of SNSs, it is these types of sites 
(Tik-Tok, Instagram, Facebook) that have become the most popular sites in cyberspace. 

In this text, the authors seek to highlight not only the scale of the PU SNS phe- 
nomenon, but also to show to what extent PU SNS is related to parental influence and 
the risk behaviour of adolescents. This research is unique because to date no such anal- 
yses have been conducted with respect to Czech adolescents in the narrow field of PU 
SNS. The research is an extension of analyses conducted in the same area, but in other 
countries [17-20]. 


3 Methodology 


3.1 Aimof the Research and Research Problems 


The aim of the research is to show the scale of Problematic Use of social networks and 
the issues that differentiate this phenomenon according to sociodemographic variables. 
The indirect aim is to determine the level of influence of risk factors (risky behavior 
in the offline space) and protective factors (restrictive upbringing methods in the home 
environment) on PU SNS. The subject of the research are the declarations of adolescents 
from the Czech Republic. The following research problems are assumed: 


RQ1: What is the level of Problematic Use of Social Networking Sites among adolescents 
in the Czech Republic? 

RQ2: To what extent are indicators of the level of Problematic Use of Social Networking 
Sites mutually related? 

RQ3: To what extent is the level of Problematic Use of Social Networking Sites 
differentiated by sociodemographic variables? 

RQ4: What is the relationship between problematic social network use and online risk 
behaviors with the application of a restrictive new media parenting style in the family 
environment? 
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3.2 Research Tools 


The study used a triangulation of quantitative research tools. The research tools were 
translated into Czech and then transferred into digital form. The battery of tests consisted 
of the following research tools: 


1. SNS use frequency scale [15], consisting of 5 items measuring SNS use style at 
mealtimes and just before bedtime and after waking up. Responses were constructed 
on a 5-point Likert scale ranging from 1 - never (low SNS use) to 5 - daily (high 
SNS use). 

2. Parental control of online activity [21, 22] constructed from 7 items, using a Likert 
scale from 1 - never (no parental control of digital safety) - to 5 - very often (high 
level of parental control of digital safety). 

3. Scale of psychoactive substance use [23] constructed from 5 items, Likert scale from 
1 - never (no experience of psychoactive substance use) - to 5 - very often (many 
systematic experiences of psychoactive substance use). 


The instrument also included basic sociodemographic questions such as age, gender, 
place of residence, and type of school attended. The questionnaire also included a field 
related to the family’s subjective sense of wealth, which was inspired by similar research 
on digital risk behavior [24]. 

The tool was translated by experts with many years of experience in conducting 
research on risk behaviour in online and offline spaces. The battery of tests was adapted 
to the age of the adolescents. The adaptation of the tool took place through pilot studies 
as well as through feedback offered by two experts involved in the research project. 
These tools included in the test battery are widely used to measure risk behaviour in 
selected Central and Eastern European (CEE) countries. However, this study did not 
aim to standardise tools on representative samples. 


3.3 Research Procedure 


The research was conducted using an online survey. The original versions of the tool 
were translated from English into Polish and Czech. The translations were completed 
by a Polish and Czech team of researchers specializing in media pedagogy. The research 
was planned in cooperation with scientists from Palacky University in Olomouc and Ped- 
agogical University in Cracow. The research was conducted among Czech adolescents 
during the school year 2018-2019 by volunteers cooperating with the Centre for the 
prevention of risky virtual communication Faculty of Education of Palacky University 
in Olomouc (PRVOK). 
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3.4 Selection and Characteristics of the Research Sample 


The research was carried out across the whole Czech Republic. The data were collected 
by persons cooperating with PRVOK. The research was conducted in the school year 
2018/2019. The respondents filled in electronic forms in the school institutions, with 
531 respondents being surveyed. Of the respondents, 58.6% were girls and the remain- 
ing 41.4% were boys. The respondents were adolescents aged 13-19 years (mean = 
16.61, median = 17, standard deviation = 1.76). The research was conducted among 
adolescents living in: villages (46.9%), towns up to 50 thousand citizens (37.7%), cities 
from 50 to 100 thousand (7.7%), cities over 200 thousand (5.8%), cities from 100 to 200 
thousand (1.9%). The respondents described the subjective status of family wealth as: 
medium (76.8%), wealthy (13.0%), low status (10.2%). The study included students who 
were educated in: secondary school that offers the maturity exam (N = 426, 80.22%), 
primary school (N = 99, 18.64%), secondary vocational school without the maturity 
exam (N = 6, 1.12%). 


3.5 Research Ethics 


The research was conducted following the usual ethical principles for social sciences. 
The questionnaire study was approved by PRVOK staff. The questionnaire was screened 
for offensive questions and data collection was sensitive to the respondents. The tool 
was devoid of items to identify respondents. Participation in the study was completely 
voluntary. The respondents could opt out of completing the questionnaire at any time. 
Participation in the research required a multi-step consent: school management first, and 
then the respondent. The research tool was preceded by a letter of introduction showing 
the purpose of the research, as well as the how the collected data would be processed. 


4 Results 


4.1 RQI1: Level of Problematic Use of Social Networking Sites among Adolescents 
in the Czech Republic 


Based on the data collected, it can be observed that Problematic Use of Social Networking 
Sites most often takes the form of SNS use as the last activity before bedtime. In this 
case, half of the respondents declare that SNS use is one of the last activities undertaken 
before bedtime. On the other hand, just after waking up, almost a third of the respondents 
also check new SNS resources with very high frequency. Almost every fifth teenager 
regularly checks SNS resources while eating dinner, while during other meals 11.3% do 
so during breakfast, and 13.2% do so very frequently during lunch. A detailed summary 
of the responses related to the frequency of SNS use during meals, just after waking up, 
and before going to bed is summarized in Fig. 2 below. 
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How often do you use social media during dinner? How often do you use social media 15 minutes before going to sleep? 


How often do you use social media 15 minutes after waking up? How often do you use social media during lunch? How onen ao you use sociai meaia auring preakfa 
N: 531 N: 531 N: 531 N: 531 N: 531 
Mean: 3,435 Mean: 2,915 Mean: 2,734 Mean: 4,137 Mean: 2,360 
Median: 4,000 Median: 3,000 Median: 2,000 Median: 5,000 Median: 2,000 
Min: 1,000 Min: 1,000 Min: 1,000 Min: 1,000 Min: 1,000 
Max: 5,000 Max: 5,000 Max: 5,000 Max: 5,000 Max: 5,000 
L-Qrt: 2,000 L-Qrt: 2,000 L-Qrt: 2,000 L-Qrt: 4,000 L-Qrt: 1,000 
U-Qrt: 5,000 U-Qrt: 4,000 U-Qrt: 4,000 U-Qrt: 5,000 U-Qrt: 3,000 
Variance: 1,944 Variance: 2,044 Variance: 1,735 Variance: 1,345 Variance: 1,902 
SD: 1,394 SD: 1,430 SD: 1,317 SD: 1,160 SD: 1,379 
Std.Err: 0,0605 Std.Err: 0,0620 Std.Err: 0,0572 Std.Err: 0,0503 Std.Err: 0,0599 
Skw: -0,327 Skw: 0,146 Skw: 0,369 Skw: -1,275 Skw: 0,677 
Kurt: -1,285 Kurt: -1,350 Kurt: -1,096 Kurt: 0,597 Kurt: -0,858 
95% Conf SD 95% Conf SD 95% Conf SD 95% Conf SD 95% Conf SD 
Lower: 1,315 Lower: 1,348 Lower: 1,242 Lower: 1,094 Lower: 1,301 
Upper: 1,484 Upper: 1,521 Upper: 1,402 Upper: 1,234 Upper: 1,468 
95% Conf Mean 95% Conf Mean 95% Conf Mean 95% Conf Mean 95% Conf Mean 
Lower: 3,316 Lower: 2,793 Lower: 2,622 Lower: 4,039 Lower: 2,242 
Upper: 3,554 Upper: 3,037 Upper: 2,847 Upper: 4,236 Upper: 2,477 


Fig. 2. Descriptive statistics - distribution of responses for the variable Problematic Use of Social 
Networking Sites 


Using the k-means cluster analysis technique, four major groups can be distinguished 
according to the intensity of social media use. Cluster number 4 (pink) are those who 
use social media in all situations. Cluster 4 (N = 28.06%) includes almost a third of 
the respondents. These are the young people who actively use new media regardless of 
circumstances. The converse situation is seen in Cluster 2 (N = 135, 25.42%), where 
one in four of the surveyed teens use SNS at a very low intensity. Cluster 2 confirms the 
stereotype that not all individuals from the younger generation are immersed in social 
media activities without interruption. In contrast, Cluster 1 (N = 100, 18.83%) and 
Cluster 3 (N = 147, 27.68%) show that the adolescent group is strongly heterogeneous 
in terms of situations related to frequent SNS use. Nevertheless, it should be emphasized 
that what almost 75% of the adolescents have in common is their very frequent or 
frequent SNS use just before bedtime. The detailed distribution of the cluster analysis 
is presented in Fig. 3. 
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-æ Cluster 1 
Æ- Cluster 2 
— Cluster 3 
-A Cluster 4 


Fig. 3. Cluster analysis by k-means: adolescent groups by frequency of SNS use 


Obviously, the five indicators indicated do not cover the full range of problematic 
situations related to SNS. It is important to include in further studies more variables 
related to the actual time use of SNS. The issue of limitations resulting from the applied 
research tool is discussed in Subsect. 4.1. 


4.2 RQ2: Co-occurrence of Indicators of Problematic Social Network Use 


To analyse the co-occurrence of indicators of Problematic Use of social networks the 
rho-Spearman correlation coefficient was used. All 5 indicators are positively correlated 
with each other. The co-occurrence is characterized by medium or strong strength. This 
means that adolescents who have adopted the habit of using social networks very often 
repeat the same behavior in different situations. For example, respondents who declare 
that they happen to browse SNS resources during breakfast also very often repeat this 
behavior during dinner or lunch. The results of the correlation coefficient are an important 
indication for the process of prevention of risky behaviors associated with new media. 
Table | below presents the results of the correlation test between the indicators. 
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Table 1. Correlation coefficient for indicators of problematic social network use 


1 2 3 4 


1 How often do you use social media 15 min | - 
after waking up? 


2 how often do you use social media during | .4038*** | - 
dinner? 


3 how often do you use social media during | .4318*** | .6634*** |- 
lunch? 


4 how often do you use social media 15 min | .5509*** | .4531*** | 4327*** |- 
before going to sleep? 


5 how often do you use social media during | .4335*** | .4556*** |.6467*** | .3264*** 
breakfasts? 


4.3 RQ3: Level of Problematic Social Network Use Versus Sociodemographic 
Variables 


The differentiation of behaviors related to problematic SNS use was also analyzed for 
changes according to sociodemographic characteristics. Based on the one-way analysis 
of variance, it was noted that slightly higher saturation of this type of problematic 
behaviour occurred among girls than among boys. Such a regularity is noticeable for all 
five indicators. A detailed visualization of gender differences is presented in Fig. 4. 


Wilks lambda=,96349, F(5, 525)=3,9784, p=,00150 


5,0 
4,5} 
4,0} 
3,5 + 
3,0 È 
2,5} 4 
Æ How often do you use social media 15 minutes after waking up? 
2,0 | How often do you use social media during dinner? 
Æ How often do you use social media during lunch? 
=Æ How often do you use social media 15 minutes before going to sleep? 
15 € How often do you use social media during breakfasts? 


girls boys 


Fig. 4. Problematic use of SNS and gender 
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It was also observed that the subjective wealth status of the family is not related to 
the level of the discussed e-risk, with Wilks lambda = .98983, F(10, 1048) = .53704, p 
= .86461. However, it was observed that a slightly higher level of saturation of selected 
problematic behaviors related to social media is found among students from secondary 
vocational schools that do not offer a high school diploma (in particular, browsing social 
media resources during breakfast). A detailed depiction of co-occurrence is presented 
in the Fig. 5 below. 


6,0 


5,5 + 


5,0 + 


45} 


4,0} 


3,5 


3,0 } 


2,5} 


2,0 } 


= How often do you use social media 15 minutes after waking up? 

1,5 | = How often do you use social media during dinner? 

= How often do you use social media during lunch? ad 

10} = How often do you use social media 15 minutes before going to sleep? 
j =- How often do you use social media during breakfasts? 


0,5 


High school with diploma Elementary Vocational secondary school 


Fig. 5. Problematic use of SNS and school type 


Metric age does not correlate with any of the indicators and thus was not deemed 
statistically significant. 


4.4 RQ4: Predicting the Level of Problematic Social Network Use Versus Offline 
Risk Behaviors and Parental Control 


In the present research, it was assumed that it is possible to predict problematic SNS use 
by considering the scale of risky behaviors in the offline space, as well as the forms of 
education for safe use of new media in the family environment. Separate calculations 
were performed for each form of problematic behavior use using multivariate regression 
analysis. Firstly, the collected data clearly show that combining risky behaviors from 
the online and offline sphere do not always correlate. In the present predictive models, 
three such cases were noted in reference to alcohol consumption, which determines the 
increase of selected forms of problematic SNS use. Moreover, in this age group, the most 
effective way to reduce the level of problematic SNS use is parental control over the time 
spent online. Since there is a different intensity of PU SNS for each of the behaviours 
identified in the tool, it was decided to carry out a predictive analysis separately for each 
factor. However, it should be clearly emphasized, referring to Table 2, that these are not 
indicators that have a strong impact on all adolescents. 
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Table 2. Predictors of problematic SNS use versus offline risk behavior and online parental control 


How often do 
you use social 
media 15 min 


How often do you 
use social media 
during dinner? 


How often do 
you use social 
media during 


How often do 
you use social 
media 15 min 


How often do 
you use social 
media during 


after waking lunch? before going to | breakfasts? 
up? sleep? 
B SE B SE B SE B SE B SE 
Cigarette 0,03 | 0,06 0,02 0,06 0,03 0,06 0,11 0,06 0,06 0,06 
smoking 
Alcohol 0,13 | 0,07 | —0,08 0,07 0,13* | 0,07 | —0,05 0,07 0,17* | 0,06 
consumption 
Getting drunk 0,03 | 0,07 0,18* 0,07 0,00 0,07 0,12 0,07 0,00 0,07 
Smoking —0,02 | 0,06 0,02 0,06 0,08 0,06 | —0,01 0,06 0,07 0,06 
marijuana 
Drug use 0,05 | 0,05 0,07 0,05 0,08 0,05 | —0,01 0,05 0,07 0,05 
Parents check if | —0,01 | 0,07 | —0,08 0,07 | —0,12 0,07 | —0,03 0,07 | —0,03 0,07 
I use the 
Internet safely 
Parents check 0,03 | 0,07 0,07 0,07 0,11 0,07 0,06 0,07 0,06 0,07 
what I do on 
the Internet 
Parents control | —0,05 | 0,05 | —0,16*** | 0,05 | —0,06 0,05 | —0,05 0,05 | —0,11* | 0,05 
the time I spend 
on the Internet 
My parents 0,04 | 0,05 0,03 0,05 | —0,01 0,05 | —0,07 0,05 0,01 0,05 
control the 
software I 
download 
Parents control | —0,04 | 0,06 0,02 0,06 | —0,07 0,06 | —0,13* | 0,06 0,05 0,06 
the games I 
play 
My parents —0,07 | 0,07 0,01 0,07 0,02 0,07 0,03 0,07 | —0,04 0,07 
control the 
websites I visit 
I have parental 0,05 | 0,06 | —0,02 0,06 0,03 0,06 0,00 0,06 0,02 0,06 
control 
software 
installed for my 
Internet 
activities 
R =,212 R2 R =,252 R2 =,063 | R =,270 R2 R =,247 R2 R =,310 R2 
,045 F(12,518) = 2.94 p | ,073 ,061 F(12,518) ,096 
F(12,518) = < .0005 Std. Error | F(12,518) = =2.80p< F(12,518) = 4.61 
2.04 p < .0190 | of estimate: 1.3991 | 3.40 p < .0000 | .0009 Std. Error | p < .0000 Std. 
Std. Error of Std. Error of of estimate: Error of 
estimate: estimate: 1.2828 | 1.1368 estimate: 1.3261 
1.3782 


* < 0,05, ** < 0,001, ***< 0,0001. 
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5 Discussion 


Adolescents most often use social media just before going to bed and just after waking 
up. Based on the data collected, it was noted that three-quarters of the respondents 
engage in such activity in the evening very often or every day. This behavior is dictated 
by the desire to access information in digital form, thus satisfying a basic need in the 
information society. SNS sites also allow adolescents to keep up to date with information 
shared by peers and peer groups, and to communicate with those close to them. Thus, 
the issue of frequency of SNS use, on the one hand, is a factor with which to identify 
problematic Internet use, while on the other hand, it is a factor that has a close relationship 
with the social needs of adolescents [25-27]. As part of the answer to the first research 
problem, it was also noted that young people very often use SNS during selected meals. 
This is also one of the factors that can be considered as inappropriate use of new media 
[28, 29]. Such behaviour is increasingly becoming the norm, which is sometimes more 
often criticised by older people, e.g. parents, while among some young people it does 
not raise serious objections linked to savoir-vivre, or is treated as something that needs 
to be corrected [30, 31]. Analyzing the scale of the phenomenon, it was also noted that 
not all young people use SNS extensively in all of the situations mentioned in the study. 
Based on the data collected, only about a third of the respondents are up-to-date (all the 
time) with social network resources. This result is more or less consistent with studies 
conducted in other countries among adolescents [32]. This means that not all young 
people are characterized by high levels of problematic Internet use. The results send a 
clear message that treating all young people as internet and smartphone addicts is an 
abuse caused by the stereotypes circulating in public opinion [33]. 

The collected results of the research confirm the previous results on different forms 
of problematic Internet use. Regardless of the adopted diagnostic factors (and the tools 
used to measure them) and measurement scales, there is a high probability that the 
indicators measuring the analyzed phenomenon mutually co-occur [34, 35]. Of course, 
co-occurrence is not always characterized by high strength of correlation for all diag- 
nostic indicators. The diagnosis of the PUI scale may therefore take a different form 
depending on the adopted reasons for new media use, time, type of devices used, and 
many other elements related to the style of digital media use. 

The findings among Czech adolescents do not differ significantly from analyses 
from other countries. Girls are slightly more likely to have higher levels of problematic 
SNS use than boys. However, this relationship is variable in the context of problematic 
Internet use depending on the diagnostic criteria adopted, as well as the type of behaviour 
commonly classified as ‘Internet addiction’. In this respect, particular caution should be 
exercised in terms of attributing the behaviours in question to gender. Moreover, when 
analyzing the elevated level of problematic SNS use, it is necessary to link the given 
indicators to the specific needs they satisfy [36-39]. 

Adolescence is a marked by a tendency to test external limits, such as those set by 
teachers and parents. Many of these behaviours are treated as typical for this develop- 
mental period. Others, on the other hand, are treated as deviations from social norms 
and are sometimes a source of intergenerational conflict. Maturation involves redefin- 
ing the boundaries set by adults [40, 41]. Earlier parenting strategies (e.g., those used 
in late childhood) related to setting boundaries regarding the use of new media, e.g., 


Problematic Use of Social Networking Sites Among Adolescents 75 


controlling the websites visited or using software that limits the time of new media and 
access to selected websites, do not meet the assumed goals for adolescents’ functioning 
in cyberspace. The results of regression models for all indicators of problematic SNS 
use clearly indicate that only for a selected group of adolescents is parental control of 
time spent on the Internet a factor reducing the intensity of SNS use. Therefore, in order 
to conduct prevention of behaviors related to problematic Internet use, it is necessary to 
move away from the assumption that only restrictive methods of education to new media 
are able to increase the level of digital safety of adolescents and protect this group from 
the harmful effects of new media [42-44]. Increasingly, it is not so much parental control 
as universal skill development that is becoming key to achieving a reduction. The preva- 
lence of high levels of PUI is seen primarily among young people who have problems 
with self-control or who have unclear boundaries for SNS use in the home environment 
[45-47]. Prevention of risk behaviors in cyberspace should therefore capture a set of 
individual and social determinants. 


6 Research Limitations 


The data collected are characterized by several limitations. Firstly, the sample was 
selected in a near random manner. It is not a study based on full and stratified sam- 
pling considering the size of the different regions in the Czech Republic. This means 
that the data are not characterized by a structure that allows generalization of SNS prob- 
lematic use behavior among all adolescents in the Czech Republic. Moreover, the data 
were collected in the pre-pandemic period, so the style of SNS use in this period is 
likely to differ significantly from the pandemic period, where a definite increase in the 
frequency of use of various e-services, including SNS, has been observed [48, 49]. 

Research on styles of SNS use is evolving. There is now increasing attention to 
issues related to the prevention of problematic Internet use, which includes problematic 
SNS use. When analyzing this type of behavior among adolescents, it is necessary to 
look more precisely at the different types of time spent using this type of media (screen 
time, awake time, thinking about SNS), as well as the real impact of SNS abuse on the 
quality of life in the offline space. Moreover, screen time should be much more precisely 
measured using improved indicators than the classic Likert scale. Real time measured 
through embedded applications (e.g. parental controls in Android systems) may become 
an effective solution in this case. Another interesting issue related to the Problematic Use 
of SNS is the connection between activities in SNS and the real needs of adolescents. 
Narrowing the research only to inappropriate locations and times of day may lead to 
partially accurate conclusions characterized by the error of reductionism. 

It should be clearly noted that the research tools used are based on questionnaires 
that have not been validated in the Czech Republic on a representative sample of adoles- 
cents. Therefore, it is important to be aware that the present research is characterised by 
limitations resulting from the lack of a full adaptation of the tool to cultural and linguistic 
conditions. Such an action will be taken in the next stages of research on problematic 
Internet use. 
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7 Conclusions 


The present research is an attempt to look at adolescents’ style of new media use. The 
research allows us to look at problematic SNS use from the perspective of times of day, 
places, and frequency of use, as well as the influence of parental control and selected 
forms of risk behaviors in the offline space. The research was conducted in the pre- 
pandemic time, so it is interesting from a historical perspective, but still reveals some 
noteworthy trends. First, SNSs have become a natural tool for adolescents to interact 
with each other [50, 51] much like analogue phones used to be, and which were likewise 
heavily used by adolescents at times and for periods that were unacceptable to their 
parents and often caused concern or conflict. The difference between the state of the 
analogue period and the current stage of development of the information society [52] is 
mainly related to the ease of access to the smartphone, multimedia, hassle-free use, and 
the effect of rewards for using new media (including SNS) [53]. Research on usage styles 
on the ways in which SNS, or more broadly digital services, are used among adolescents is 
an interesting area of research on the culture and ways in which young people function in 
cyberspace [54]. In turn, questions about the scale of the phenomenon and the protective 
and risk factors associated with problematic SNS use are issues that relate directly to 
dynamically changing diagnostic criteria and individual, cultural conditions and global 
processes [55-58]. 
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Abstract. The present study was aimed to investigate English learning as second 
language, in school, in first, second and third graders of twelve classes randomly 
assigned to a control or an experimental group. Children in the latter are exposed 
during English school teaching to the method “An English Island®” and to its 
platform activities. The method “An English Island®” offers a variety of strategies 
for teaching English in primary school, an innovative digital tool that promotes 
teaching/learning English language’s communicative approach, lead students to 
become familiar with the language in a sort of continuous, inclusive workout, in 
which everyone participates and talks. 

English skills as well as cognitive abilities are tested in both groups at the 
beginning and at the end of the school year with the aim to compare control and 
experimental classes in both a longitudinal and a cross-sectional design. 


Keywords: English learning - English teaching - Primary school - English 
communicative skills 


1 Introduction 


According to the M.I.U.R. National Guidelines for nursery and primary schools (Septem- 
ber 2012, p. 37), “in learning languages, motivation comes from the students’ natural 
aptitude to communicate, socialize, and interact as well as from their willingness to 
“exercise their tongues” [1]. 

Thus, the communicative aspect prevails when the starting point is the oral language. 
Because of the lack of linguistic transparency in English, the written language gets in 
the way, causes errors, and creates disparities in students with SLD, Specific Learning 
Disabilities, or other problems [2]. Writing, especially in primary school, means speaking 
on paper, and it is about arriving at spelling only after being spoken correctly. 
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In order to keep the children’s motivation alive and steady, it is important to use 
multi-sensory strategies, presenting information through different methods. In addition, 
the effectiveness in the learning phase of integrating sensory input through other channels 
is now recognized. 

Factors such as stress and anxiety are reduced by increasing communication skills, 
motivation, and self-esteem, highlighting that “both activity and direct personal experi- 
ences—games, physical activities—play a fundamental role in children’s development 
and early learning” [3]. 

The different phonemes and graphemes between the English and Italian languages 
should always be considered so as not to create difficulties in understanding and speaking 
[4]. The child must become familiar with the sounds and sound differences always asso- 
ciated with the meaning/phoneme through conscious and unconscious assimilation” [5]. 
Consequently, the language becomes proactive communication, with “language learn- 
ing based on the children’s recognizing their ability to communicate with others, their 
sensitivity to visual attention, and a desire to imitate” [6]. 

The method “An English Island®” uses a series of didactic strategies to transform the 
role of the teacher, who becomes coach and observer, and directs the learning process, 
focusing closely on the learner role (learner-centred) and the learning process (process- 
oriented instruction). 

The “An English Island®” methodology develops a “circular learning path” that 
activates three expressive languages (mime, visual, and oral) to fix almost simultaneously 
the elements by association. The synergy of these three languages helps to create the 
mental/physical grooves that, developed over time, expand through habit and repetition 
and help to create familiarity. Thanks to this mental groove, created through mime and 
drawing, the child can globalize the written word with the oral one and read it correctly. 

Thus, written language is introduced gradually, as shown by recent studies: “Among 
the multiple language competences to be pursued, writing is definitely the lowest pri- 
ority in primary school, also in consideration of specific learning difficulties (dyslexia, 
dysgraphia, dysorthography) that usually emerge at this school level and that may be, in 
some way, also increased in the mother-tongue by an approach to English which focuses 
too much on writing” [7]. 

An innovative approach to teaching and teaching foreign languages cannot ignore 
the issues of inclusion and the personalization of teaching-learning paths. The method 
“An English Island®” offers a variety of strategies for teaching English in primary 
school that, through codified modalities, seek to achieve the same goal: to develop oral 
communication skills in all children. 

“The use of the spoken language should be introduced by the teacher in a systematic 
way, proceeding gradually according to the complexity of the language structures pre- 
sented” [7]. On the other hand, speaking English for the child has to become a natural 
activity, preferably integrated with the use of creative and interactive games that can be 
supported by other nonverbal languages (music, movements, images, etc.) to promote 
participation and involvement in cooperative learning activities [8, 9]. 

Finally, creating a learning environment that responds to metacognitive teaching 
canons will enhance the development of a generation of people who will know how to 
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navigate in a constantly and unpredictably changing panorama of life by being effective 
problem solvers and lifelong learners. 


2 Learning English as Foreign Language 


The research on foreign language learning suggested that it is a complex skill, largely 
based upon native language proficiency. Sparks and Ganschow [10, 11], for example, 
analysing learning and cognitive profiles of high school and college students, showed 
that foreign language learning is built on phonological-orthographic and syntactic skills 
in native language, according to the Linguistic Coding Difference Hypotesis - LCDH 
they developed following their evidence. LCDH predicts that students with weaker 
native language skills, particularly in grammatical knowledge and sensitivity as well 
as sound-symbol association and language phonology, may show poor foreign language 
proficiency [12]. 

The involvement of L1 processing in FL learning was confirmed by several stud- 
ies conducted within different languages. For example, a series of studies conducted 
on Finnish young student, learning English as foreign language, demonstrated that 
phonological-orthographic ability in L1 and phonological memory are prerequisites for 
learning English as foreign language [13-15]. Moreover, an association between native 
reading decoding skills and second language learning has been observed. FL proficiency 
was explained by the ability of word reading in L1 [16]. Similar results were obtained 
with a group of college students: The authors found that the native language decoding 
skills predicted students’ second language proficiency [17]. 

Some studies have been conducted with Italian students learning English as foreign 
language. Young Italian students with Difficulties in Learning English as Foreign Lan- 
guage (FLLD) were compared to control students without difficulties [18] on memory 
measures both verbal and visuo-spatial. Results indicated that poor components of verbal 
working memory are related to students’ foreign language learning difficulties. These 
students showed also weak ability in phonological sensitivity [18], again verbal skills. 
Indeed, phonological sensitivity, a broad terms that refer to the analysis and manipu- 
lation of the sub-components of spoken word forms, including specific processes such 
as blending, deleting, and synthesising the speech units [19] seemed to be relevant not 
only to native language acquisition and reading abilities [20, 21] but also contribute to 
a correct analysis, representation, and manipulation of a new phonological trace during 
foreign language learning [22]. However, it should be noticed that due to the high trans- 
parency of Italian ortography, Italian readers can use effectively a phonological strategy 
(i.e. a direct grapheme to phoneme conversion) but, such phonological strategy seems 
not effective in English, due to its low orthographic regularity [23]. Palladino et al. [24, 
25] demonstrated how children with dyslexia may have trouble in learning to read and 
to write in English as second language. However their performance were not completely 
negative since reading non-words appeared comparable to peers without dyslexia and 
in spelling errors were mainly phonological. Non phonological errors were comparable 
between same age groups, with or without dyslexia. 

In the Italian context, the article 5 (point 2c) of the law 170/2010 describes the 
devices that schools have to adopt for teaching FL to student with specific learning 
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difficulties. Beside the use of compensatory tools promoting oral communication and 
ensuring a gradual learning, the law allow the possibility of exemption. In addition, it is 
recommended to choose, if possible, a transparent second language. However, this kind 
of policies (i.e. exemption and the choice of a transparent FL) seem disadvantageous 
for students with learning impairments. Research demonstrates indeed that in a more 
immersive environment, with an appropriate balance between literacy and communica- 
tion, teaching may be really inclusive and learning may be promoted for all students 
[26]. 

Miller-Guron and Lundberg [27] discovered adults with dyslexia with a preference 
for reading in English: they were Swedish adults able to learn English at school at such 
a good level to be able to feel so confident in using the second language for reading. 


3 Why the “An English Island®” E-learning Platform was Created 


There is increasing discussion of “digitization”, as highlighted in the National Plan for 
the Digital School. “It is, therefore, necessary that digitization efforts be channelled into 
the idea of innovation, of a school no longer solely transmissive, and an open, inclusive 
school in a changing society... At the same time, staff training geared towards didactic 
innovation and openness to organizational innovation will be crucial for taking a step 
forward. Education in the digital age must not focus on technology but on the new 
models of didactic interaction that use it. Moreover, teachers, particularly with regard to 
digital skills, must have the right conditions to act as facilitators ofinnovative educational 
courses based on more familiar content. Teacher training must be centred on didactic 
innovation, taking digital technologies into account as a support for implementing new 
educational paradigms and the operational planning of activities” [28]. 

Alongside this important prospect is Legislative Decree no. 62 of 13 April 2017, 
which introduced important changes in student assessment, including the INVALSI 
tests. Over the years, these tests have shown different, although positive on average, 
results in the three Italian macro-areas, starting with primary school. However, a closer 
look shows the importance of focusing on the degree of equity in the educational system. 
While it is stated that “Schools should ensure the same educational opportunities for all, 
without offering more fortunate students because of their family or social background 
the best possibilities, and vice versa”, the results show that “the variability component 
between schools and between classes is higher in southern Italy and the islands than in 
central and northern Italy. This means test results are not only lower overall in the south 
and on the islands; they also differ more from one school to another and from one class 
to another in contrast to the rest of Italy. Therefore, the school system is less effective in 
the south and less capable of ensuring students have the same educational opportunities” 
[29]. 

The 2021 INVALSI tests confirm that “Primary-school outcomes are substantially 
stable, but the considerable difference between schools and classes in the south and the 
rest of the country remains.... Very important differences were already found within 
the school system in the first analysis of the data, not only for the territories but also 
depending on the original social environment” [30]. 

In light of these challenges, the British School-Pistoia created the e-learning plat- 
form “An English Island®” as an innovative and effective digital tool that homogenizes 
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teaching/learning both vertically and horizontally between the various classes and the 
various schools. This facilitating tool for teachers conveys the English language’s com- 
municative approach through a series of teaching strategies that, in various forms, lead 
students to become familiar with the language in a sort of continuous, inclusive workout, 
in which everyone participates and talks. The main goal is to ensure that all students are 
included, even those most disadvantaged from a socio-economic-cultural point of view, 
along the democratic course of learning that must take place in public schools. 

Furthermore, and by no means secondary, if skills/knowledge in the primary school 
can be increased, there will be a push upwards, and fertile ground will be further cultivated 
in the lower and upper secondary schools, improving current results and providing a 
passe-partout for active citizenship. 

The innovative e-learning platform “An English Island®” was designed and con- 
structed to offer primary school classes 1 to 5 an educational path, divided into units, 
complete with suggestions, strategies, audio, videos, multimedia, and printable mate- 
rials. In addition, the use of educational content can be conveyed through an IWB, 
Smart-TV, and other devices to make students’ participation in this learning program 
even more appealing. 

The knowledge and skills developed and acquired over a five-year period meet the 
corresponding goals of the A1 Level of the Common European Framework for Languages 
(C.E.F.R.) of the Council of Europe. 

In programming the platform, particular attention was paid to teachers in their role 
as educational facilitators and who, therefore, must become familiar with the structural 
system, the founding principles, and its applications. Therefore, a Training area on 
guidelines and strategies for “An English Island®” was produced. Moreover, it is always 
available for a self-directed consultation through a series of 11 podcasts that can be used 
immediately to develop an in-class teaching activity and facilitate the coach’s role. Over 
time, on-the-job training accompanies the teacher, whose support and advice become a 
personal reference for resolving doubts or oversights and optimal planning of activities. 

Not only is the teacher’s scaffolding action intellectual or technical [31], it is also 
emotional, cognitive, and metacognitive. The emotional sphere concerns stimulating 
the student to learn through encouragement and overcoming motivational barriers. The 
development of metacognitive skills enables the child to learn continuously and confront 
more complex processes such as reflection [32]. An awareness of one’s progress and 
being equipped with the required skills help to increase the student’s self-esteem. 


4 Structure of the E-learning Platform “An English Island®” 


This digital platform was developed to provide practical support for teaching children 
to speak English through continuous practice. Its focus on oral communication makes it 
a practical tool that integrates curricular teaching. 

Each platform step has video and audio made by native speakers, cards with codified 
drawings that move when clicked and speak individually or move by building the various 
sentences by pronouncing them, following the right rhythm. With the same video game 
appeal, children are excited about participating, acting with the body, using cards, writing 
through drawings, and touching the IWB until they become protagonists and teachers 
themselves. 
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Five levels have been created, one for each primary school class, modelled on the 
same structure scheme repeated for all 48 units, inserting different and progressively 
more complex contents in line with the school program. 

The teacher has a general index representing the year’s activity and contents to enter 
the various units directly. Each is introduced by a mission that the teacher presents to 
the class, focusing on motivation and consequent problem-solving. 

Each unit is developed with the following series of learning strategies: Mimes, Board 
Game, Circles, Twin Brother, Initials (from class 3 onwards), and Mix & Match. 

Mimes: 

This engaging strategy associates three languages: mime/visual/oral. It captures the 
students’ attention and enables increasing their participation as they become conscious 
actors. 

Board Game: 

It is the most powerful strategy: through the drawing cards, grouped by chunks, 
which “speak” and are moved by clicking on them, the students construct the sentences 
without any written words yet. The exercise uses the drawing cards already associated 
with Mimes and repeats orally in different ways what has already been learned, forming 
complete sentences. 

Circles: 

A listening comprehension exercise: a layout is used where the drawing cards of 
the grammar structure are positioned at the top, and the empty circles, representing the 
second part of the sentence, are at the bottom. The teacher plays the sentence several 
times, and the students then draw what they hear in the circles below. By clicking on 
the circle, the corresponding drawings appear. Therefore, it is possible to self-correct. 
(Written words have not been introduced yet!). It is an inclusive, SLD-friendly exercise. 

Twin Brother: 

This exercise associates the oral word with its spelling and is the first encounter with 
written words. Despite this, students are confident because they have acquired familiarity 
and habit via the various modes of repetition earlier. 

Initials: 

Initials is an aid, a kind of visual guide, that helps to understand how many words 
are contained in a chunk or a sentence. It also helps to avoid grammatical mistakes. They 
are the last step to arrive at the written word now that oral familiarity has been acquired. 
This exercise is seen as a game by all the students who view it as a sort of competition 
to recognize the sentence and then say it. It is first introduced in the third grade. 

Mix&Match: 

It trains for sequential or random listening comprehension of sentences spoken by 
native speakers identified in the list and heard both in English and Italian to verify 
understanding and do any translation. The exercise leads to a conscious consideration 
that represents a transition from habit to knowledge. 

The platform offers teachers four specific podcasts on Extra Speaking Games 
(Cards/Pencil Game/Finger Game/Child Actors), which are played with the students to 
improve rhythm in pronouncing chunks or phrases, correct any errors without explaining 
grammar rules, and the sequential positioning of the drawing cards to build the sentences 
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respecting the rhythm. These are games that the teacher can introduce at any time with 
the IWB off and repeated by involving the students as fully as possible. 

The platform has a Dashboard through which the teacher can move from one exercise 
to another, from one unit to another, play games, suggest activities, and continuous 
listening for use in passive and active phases. In addition, the “T’ = Teacher area can be 
consulted where suggestions and tips are put forward, as well as any printable materials 
for classroom exercises. 


5 Aimsof the Study 


The present study is aimed to test the role of cognitive variables such as short term 
memory and working memory but also emotions and motivation in learning English as 
second language at school. In particular the study compared two conditions of learning: 
a control condition where teachers followed traditional textbooks and an experimental 
conditions where teachers were able to implement in the ordinary teaching activity the 
use of the e-learning platform “an English Island”. 


6 Methodology and Tools 


6.1 Design 


The study has a longitudinal design and wants to investigate whether the use of the app 
An English Island has significant effects in learning English in primary school children. 

To this end, a battery of assessment was provided to be given at two different times 
(t1 and t2) i.e. before and after the treatment. The treatment is identified in the learning 
of the English language through the app An English Island. The group was divided into 
control group and experimental group in order to assess the presence or absence of any 
significant change. 

At tl, collective tests were administered in order to investigate emotions related to 
learning English; subsequently, an individual assessment was conducted using a bat- 
tery of instruments set for the reference class (first grade; second grade; third grade). 
Currently, the study is in the individual assessment phase at t1. 

In concomitance, interested teachers are learning the use of the app. Teachers have 
in fact experienced an initial cognitive and exploratory meeting about the project’s aims 
and the educational and training objectives of the platform; currently, teachers are being 
trained through specific webinars and lessons built to meet the needs of the teachers and 
children involved. 


6.2 Participants 


The participants of the study are primary school children aged between 6 and 8 years. 
The school of reference is the “Istituto Comprensivo Santa Chiara - Pascoli — Altamura” 
located in Foggia. In particular, the participants are children attending elementary school 
in the Santa Chiara and Altamura school complexes. 
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The study involves n = 12 classes of elementary school for a total of n = 193 
participants (n = 102 males; n = 91 female), the participants are distributed in control 
group (n = 102; n= 51 males; n = 51 female) and experimental group (n= 91; n= 51 
males; n = 40 female). Specifically, the control group consisted of n = 2 first classes (n 
= 23;n = 11 males; n = 12 female); n = 2 second classes (n = 40; n = 22 males;n = 
18 female); n = 2 third classes (n = 39; n= 18 males; n= 21 female). The experimental 
group consisted of n = 2 first classes (n = 30; n = 19 males; n = 11 female); n = 2 
second classes (n = 26; n = 12 males; n = 14 female); n = 2 third classes (n = 35; n = 
20 males; n = 15 female). 


6.3 Procedure 


The participants in the experimental group follow an English course through an inno- 
vative didactic method using the app An English Island. In order to compare the skills 
acquired by the experimental group with those of the control group, a collective battery 
and an individual battery are administered before and after the course, both conven- 
tional and experimental. During the screening session with collective battery, participants 
provide written informed consent before undergoing an assessment. 

Collective Test: AEQ-ES 

Achievement emotions in English as a second language were assessed by means of 
the Italian version of the Achievement Emotion Questionnaire for elementary school 
children [33] (AEQ-ES). Since the original questionnaire included only the domain 
of first language (i.e., Italian), for the present study, we adapted the questionnaire by 
substituting Italian with English. The questionnaire consisted of a total of 32 items: nine 
on enjoyment, 12 on anxiety, 11 on boredom. These were divided in three parts referring 
to different settings, that is, attending English class (e.g., I enjoy English class), doing 
English homework (e.g., English homework bores me to death), and taking English tests 
(e.g., When I take an English test, I am afraid of getting a bad grade). For each item 
children had to rate the intensity of their emotion on a five-point Likert scale ranging 
from 1 (= not at all) to 5 (= very much). The scale was supported by pictures of faces 
of boys or girls (matched with children’s gender) representing the intensity of emotions 
[34]. The completion of the questionnaire took a total of 45 min. In order to avoid fatigue, 
children were given the possibility to take short breaks between the different parts of the 
questionnaire. 

Individual Battery 

The individual battery consists of 7 tests that assess: learning new words, working 
memory, long-term memory, comprehension, production, pronunciation and repetition 
of English words, metacognition of the performance produced and problem-solving 
skills. Below are the tests that make up the battery. 


a) Learning New Words 
The test for learning new words is an Auditory Verbal Learning Test, where the 
participant is asked to repeat a list of 10 non-words (such as Zila, Muci and Libo), 
consisting of 2 syllables and having a total of 4 phonemes [35]. When children hear 
and then utter a non-word for the first time, they must transform a novel speech 
signal into a series of coordinated, precisely timed oral movements. The test is used 


b) 


d) 


e) 
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to measure verbal learning and is administered by reading the list of non-words to 
the child at a rate of one word per second. At the end of the reading, the child is 
asked to repeat as many of the newly heard non-words as possible in any order. This 
procedure is used, with the same list of words, for five consecutive times, each time 
recording the correct non-words recalled as well as the intruding words reported by 
the subject. The test is expected to measure learning. 
Digit Span 

The children’s version of digit span is a subtest from the WISC-IV scale [36]. The 
test consists of an immediate serial recall of a string of numbers. This test requires 
children to recall a sequence of digits in the same order (DSF) or in reverse order 
(DSB) that the experimenter has given them verbally [37]. The score corresponds 
to the highest span measure, with the sequences produced correctly at least two out 
of three times. 
Delayed Recall 

Approximately 10 min after the immediate recall test, during which time the 
digit span test is administered, the patient is asked to remember (without the list 
being presented again by the examiner) as many words as possible from the list. 
This is the second part of the test and consists of delayed recall. The test measures 
verbal long-term memory capacity. 
Repetition of Short Words 

In this test, an audio recorder is set up so that the details of the performance can 
be assessed later. The Experimenter reads to the child words in English and requires 
the child to repeat the verbal stimuli just heard. The terms used are read, turn, seat, 
mind, law, hell. Children’s production is recorded and analysed in order to score two 
different indices: Accuracy and correct pronunciation. 
English Vocabulary 

This test involves comprehension and production of the English language. The 
tests are individualized and are videotaped so that comprehension and production 
constructs can be analyzed later. All vocabulary tests begin with an initial familiar- 
ization phase to the task and are individualized with respect to the target classes. 
Rehearsals include a first part during which the child listens to the production, by 
the experimenter, and, immediately afterwards, states the corresponding picture to 
the word heard. In the second part, the child produces the word associated with the 
image indicated by the experimenter. 


— For the first grade, the assessment alternates between the comprehension task 
and the production task: the child listens, one at a time, to the following numbers 
in English “four, eight, three, six, ten, five” and points with his fingers to the 
corresponding number. Next, the experimenter points to the following numbers 
on the relevant pictures “nine, two, seven, four, eight, six” and the child verbally 
produces the corresponding number in English. In the second phase, there is 
again a comprehension task: the child listens, one at a time, to the following 
colors “yellow, green, black, orange, red, brown” and points to the corresponding 
color from the selected and presented pictures. Next, the experimenter points to 
the following colors “black, orange, brown, green, yellow, red” and the child 
verbally produces in English the corresponding color. 
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— For class two, the assessment alternates between the comprehension task and 
the production task: the child listens, one at a time, to the production of the 
following school objects in English “pencil, rubber, book, ruler, glue, pen” and 
points to the corresponding object among the pictures selected and presented. 
Next, the experimenter points to the following animal names “tiger, cat, fish, 
dog, cow, sheep” and the child verbally produces in English the corresponding 
animal from the pictures presented. 

— Forthethird grade, the assessment alternates between the comprehension task and 
the production task: the child listens, one ata time, to the production of the follow- 
ing garments in English “t-shirt, dress, shoes, jacket, coat, scarf” and points to 
the corresponding garment among the pictures selected and presented. Next, the 
experimenter indicates the following domestic environments “bathroom, living 
room, kitchen, dining room, bedroom, basement” and the child verbally produces 
in English the corresponding environment among the pictures presented. 


It should be noted, moreover, that all the images come from depository without 
copyright. 

Metacognitive Control Task 

The metacognitive control task is closely related to the English vocabulary pro- 
duction test. The test was adapted from the studies of Roebers et al. [40] and Lavis 
and Mahy [41]. In the test, the experimenter presents a 5-point confidence scale 
consisting of faces. The scale has 5 faces and represents the confidence judgment. 
Specifically, the extremes represent a very smiling face and a frowning face. The 
smiling face means “very, very sure”, the frowning face means “not sure at all”, in 
the middle there is a neutral face meaning “uncertain”. At the end of the vocabulary 
rehearsal, the safety judgment through faces is explained to the child; we verify that 
the child has understood by asking control questions such as “how old are you?’, 
“how old am IY”, “how much hair do you have on your head?” and based on the 
indication on the scale we try to understand if the child has understood the safety 
Judgment scale. Next, the pictures related to the vocabulary task are presented again 
and the child is asked to indicate how confident they were in the vocabulary they 
provided. 

Next, two boxes are presented to the child: one red and one green. The child is 
asked to put pictures in the red box if they think they said the wrong word and to 
put pictures in the green box if they think they said the right word. The purpose of 
these confidence judgments is to encourage children to reflect on the accuracy of 
their verbal production after performing the task. 

Problem Solving 

This test has problem solving skills applied in a situation where communication 
in English is necessary. The experimenter introduces the child to a situation that 
must be solved through verbal production in English. The tests are individual and 
are video-recorded in order to be able to subsequently analyze the investigated con- 
structs; moreover, the tests are individualized with respect to the reference classes. 
Each problem solving test has, in particular, two phases: in the first phase there is 
the simulation of a situation, in the second phase the child must respond, in English, 
to a question posed in English. 
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— Forclass one, in the first part, a situation is simulated in which it is necessary to 
ask another child, in English, his or her name. In the second phase, the child is 
asked, in English, how old he/she is and asked to answer in English. 

— For class two, in the first part, a situation is simulated where you want to ask 
a child, in English, who his favorite singer is. In the second phase, the child is 
asked, in English, what his favorite sport is and is asked to answer in English. 

— For the third grade, in the first part, a situation is simulated where the child is 
in an ice cream shop and is asked, in English, for a chocolate ice cream. In the 
second phase, the child is asked, in English, when his class starts. 


7 Conclusions 


The spirit that inspired the British School-Pistoia to create the “An English Island®” e- 
learning platform is connected to a vision whose fundamental core is to improve English 
language knowledge/skills as early as primary school. Pistoia is, by definition, the city 
of plants, a permanent nursery that focuses on producing strong and healthy plants. If 
some in the group of plants are a little weaker, they are supported by the others, and 
everyone together can create a wonderful forest under the banner of inclusiveness. No 
matter where they teach—classroom, school, or city—teachers can find in this platform 
an organized framework that facilitates their work, advancing the goals pursued in their 
constant daily effort to achieve our School 3.0. Our study may therefore better clarify the 
role that a teaching approach like that promoted by this e-learning platform may exert on 
learning English as second language at school in early grades of primary school and shed 
light on the complex relationships between cognitive and metacognitive processes and 
emotion and English learning at school comparing two different teaching conditions. 
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Abstract. Diligo 2.0 is a mobile app designed as a serious game to assess spatial 
and numerical cognition as key elements of the school readiness concept in asso- 
ciation with a preference for slow and fast thinking strategies. School readiness 
is a key concept for the future development of cognitive and emotional abilities 
of children, and it is highly correlated with academic success. The app is also 
focused on evaluating a preference for slow or fast thinking activities. The Diligo 
2.0 app has been developed for Android platform and has been distributed in two 
Italian schools as a pilot study with 44 children. Usage data have been collected 
and are discussed in this paper to show possible directions for this kind of digital 
tool both for assessment and for training children’s abilities. 


Keywords: Mobile app - School readiness - Assessment - Children 


1 Introduction 


Children approaching primary school are asked to meet several standards and find them- 
selves at facing several challenges when starting their entrance to the school system. In 
this scenario, the concept of school readiness becomes a fundamental factor to consider 
[1]. School readiness is a label used to describe children’s mathematical, literacy and 
socio-emotional skills that are strong predictors of their academic success and personality 
development [2-4]. 

The importance of these skills is widely recognized, and it has become a key ele- 
ment in the Italian National Guidelines for Kindergarten since 2012 issued by the Ital- 
ian Ministry of Education [5]. School readiness is a set of abilities that represents the 
basic achievements, from a cognitive and emotional point of view, to face the transition 
between kindergarten and primary school. 

Considering the different abilities which are included in the concept of school readi- 
ness, we focused on two main branches stemming from them, or numerical and spatial 
thinking and socio-emotional skills. These two cores have been used as the root for 
the development of two parallel versions of the Diligo 2.0 mobile app [6]. The version 
which focuses on emotional skills is designed to assess these key abilities: awareness 
of the emotions, use and comprehension of emotion-related vocabulary, recognition of 
facial expressions and their link to the emotions, comprehension of the situations that 
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elicit emotions, knowledge of the cultural rules for displaying emotion and regulation 
and management of one’s own and others’ emotions. The other version, which is the 
one which has already been tested in schools and whose results are discussed in this 
paper, is focused on these skills: knowledge of the geometrical figures, acquisition of 
big and small concepts, recognition of number representation, spatial concepts of in-out 
and up-down, temporal order in terms of before and after, spatial directions left and right. 
These mobile apps not only allow to monitor the school readiness abilities, but they 
also evaluate the cognitive preference for slow or fast thinking activities as described 
in Kahneman’s theory [7] who describes human thinking based on two main reasoning 
systems which uses different strategies, where “System 1” is fast, automatic, uncon- 
scious, and emotional and “System 2” is slow, logical, conscious, and effortful. So, the 
hypothesis underlying this study is that children are much more used to engage in fast 
thinking activities that in slower ones, and this may be due to the experience children 
have more chances to encounter in modern entertainment forms and environments. 


2 The Structure of the App 


Diligo 2.0 can be described as a serious game, developed for children of age 5 to 6 years 
old. The game has been developed using the STELT (Smart Technologies to Enhance 
Learning Technologies) [8] development framework and it follows an agent-based app- 
roach [9], since it is based on the continuous interaction between a natural agent (the 
kid) and the main character of the game, which is the artificial agent. 

The game is focused on helping the character of “Leo the Explorer” to solve puzzles 
in eight different islands to collect treasures. To solve the puzzles, children must answer 
to 32 different questions, divided by eight sets of items, one for every school readiness 
skill related to numerical and spatial thinking. For every interaction, the Explorer has 
been fully voiced by a professional voice actor, and if needed, the child can listen any 
number of times the instructions by just pressing an on-screen button. For every group 
of four items, there have been programmed four alternative forms, which are randomly 
selected for every game session, allowing the game to be various enough for children if 
played more than once with a total number of 128 possible items. 

At the beginning of every level, which is represented by an island on a treasure map, 
the child can choose between a slow or fast approach by selecting the preferred mode. 

The fast mode is characterized by presenting the four items of the skill in quick 
sequence, without any storytelling, encouraging the child to play as fast as possible, as 
it is shown in Fig. 1. 

The slow mode is based on a narrative approach, since the same item of the fast mode 
are mixed with other requests for the child that does not deal with numerical or spatial 
abilities but are necessary for the story undergoing in which the child and the Explorer 
are involved. The idea is that a narrative approach may improve children’s performances 
[10-13], since it activates Kahneman’s System 2, allowing the children to better focus 
on the task, and achieving a higher score in the game. It is possible to see a “narrative” 
item from slow mode in Fig. 2. 

The app can perform both online and offline, requiring an online connection to send 
data to the database where they have been stored for this study, and where they can be 
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Fig. 1. An item from the number recognition skill. In this case, the Explorer voice asks the child 
to say if the two depicted number are the same or not. 


Fig. 2. This is an example of narrative items from the slow mode. In this case, the Explorer’s 
voice asks the children to choose an item to run away from the shark during an immersion. This 
kind of item does not affect the final score. 


accessed only by researcher and in anonymous mode. The only score the child can see 
in the app is the preference for fast or slow mode, and this is because Diligo is still not 
validated as an assessment tool, so we did not want it to become misunderstood. 

The concept behind Diligo is both to create the structure for an ipsative tool and to 
make a normative one. The game can be an ipsative tool for teachers to collect cumulative 
data on a specific child, and at the same time, teachers can monitor the entire classroom 
considering it as a normative tool to understand the level of the entire group. 

Both numerical-spatial cognition and emotional skills versions of Diligo are currently 
available in Italian language for Android devices and can be found on the Google Play 
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Store [14, 15], but the one we have used for this pilot study is the one focused on the 
numerical and spatial cognition. 


3 Methods 


This pilot study involved n = 44 Italian children from kindergarten with age between 5 
and 6 years old from two different schools in Center Italy. 

Before administering the mobile app, a prototype of the app has been given to teach- 
ers, allowing them to familiarize with it, and to collect feedbacks from teachers through 
a checklist. These feedbacks have been since used to correct some minor issues and to 
improve the UI and the overall experience of the app. 

Before the experiment, teachers were given access keys to associate to children 
profiles to collect data anonymously but allowing children to login as distinct users. All 
data collected has been anonymously and safely stored in an online database. 

The experiment has been conducted with the help of teachers, who administered 
the mobile app through Android tablets in small groups, so that every child had the 
opportunity to play a complete session, receiving help from teachers if they had technical 
issues. 


4 Results and Discussion 


We can start describing results by observing Table 1, where descriptive statistics have 
been reported. Every sub/scale had a minimum possible score of 0 and a maximum of 
4 per-scale, while the total maximum score for the complete session was 32. The mean 
score of the sample is slightly above the mode, which shows that most of the children 
answered correctly to the 58% of the items. 

This may be explained by the difficulty of the items, or by the dimensions of the 
sample, which may be not enough big to shed light on this data. Even if we consider 
the possible difficulty of items, we could attribute it to different factors. Since we used 
a mobile app, UI usability, devices performances and graphics may have a major role 
in children’s performances, so it would be interesting to repeat this experiment together 
with other available evaluation methods for numerical and spatial skills. We can also 
compare the different scores in subscales. One notable difference is the low score on the 
“Inside-Outside” and “Up-down” scales, which have the lowest scores on all descriptive 
measures. 

Taking account that other spatial thinking abilities shows higher scores, it is possible 
to hypothesize that these low scores may be due to some interface or graphics factors, 
like interface readability or level design elements. This aspect may play an important 
role, since the environments of the game are realized in 2D, especially in these two 
subscales, which may benefit more than the others of a 3D environment to improve UI 
clarity. 

We can now discuss data related to the preference for fast or slow thinking modes. 
Considering Table 2, we can notice that all the measures are very much closer to 1 than 
to 2, which means that if 1 is the score assigned for the fast mode and 2 the one for the 
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Table 1. Descriptive statistics for the n = 44 children sample considering correct answers 


Subscales Mean Median Mode Std. deviation 
Shape 2,59 3 3 1,064 
Dimension 2,68 3 4 1,216 
Number 2,93 3 4 1,043 
Inside-outside 1,36 1 1 0,942 
Up-down 1,68 1 1 1,137 
Before-after 2,55 3 3 1,19 
Left/right 1,86 2 2 1,357 
Differences 2,98 3 3 1 

Total 18,64 18,5 16 4,143 


slow preference, most of the children showed a strong tendency towards the use of fast 


thinking cognitive strategies. 


This data is even more consistent when considering that standard deviation for this 
measure is quite low and even smaller than the one observed in the score for the different 
subscales in Table 1, showing that variance is low, and it is very evident the preference for 
fast mode. This result is quite close to the hypothesis, expressed in the previous paragraph, 
that today’s children may have a stronger preference for fast thinking activities due to 
the exposition to experiences and entertainment products that privilege this mode at the 


expense of slow cognitive activities. 


Table 2. Descriptive statistics for the n = 44 children sample considering fast/slow preferences, 
where 1 = fast and 2 = slow 


Subscales Mean Median Mode Std. deviation 
Shape 1,48 1 1 0,505 
Dimension 1,52 2 2 0,505 
Number 1,36 1 1 0,487 
Inside-outside 1,61 2 2 0,493 
Up-down 1,36 1 1 0,487 
Before-after 1,55 2 2 0,504 
Left/right 1,45 1 1 0,504 
Differences 1,57 2 2 0,501 
Total 1,11 1 1 0,784 
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5 Conclusions 


The first results coming from this pilot study on the usage of Diligo 2.0 open some 
interesting scenarios to better understand. 

The first point to deepen is the impact of the game design, UI and UX on the per- 
formances of children. This theme is highly relevant when designing a serious game 
which has the aim to become an assessment tool, since the user experience of the design 
elements of the app/game inevitably add a layer between the user and the evaluation tool, 
having an impact on the final performance that can be improved or slowed down by the 
ease of use and clarity of app design. It would be interesting, for example, to redesign 
the UI for the two subscales which have shown lower average scores, because it would 
allow to better understand if a 2D or 3D environment can affect children performances 
on cognitive tasks. 

Further data analysis is needed to be realized on a bigger sample, since preliminary 
data analysis showed a weak reliability of item scales (a = .61) with different method- 
ologies (KR-20 = 0.59 and KR-21 = 0.56), and this may easily be due to the small 
sample or the shortness of the scale. 

Even correlation seems to be quite low between the preference for fast mode and 
average scores on all subscales, but for the same reasons above, it is difficult to consider 
appropriately this data on a small size sample. Even a preliminary one sample t-test has 
not shown significant differences in average scores between children with preference 
towards fast and slow thinking activities, and this may be linked to the size of the 
sample, so further analysis and a bigger study are needed to better understand the role 
of cognitive mode preference and performance, which seems, by now, not significant. 

For the future development of the Diligo 2.0 app it will be important to improve 
game design, UI and UX and, moreover, it is soon expected to start a new pilot study 
with the other version of the game, which is focused on social and emotional skills. 

This may give even more information to understand the role of the design of the 
app on children performances, other than giving information about the specific skills on 
which the game is focused. 

Summarizing all these points, future studies with Diligo 2.0 need to include a bigger 
sample, to consider possible differences related to gender, and to improve the game UX 
to make it even more accessible and usable in kindergarten. 


Acknowledgements. Diligo 2.0 project has been developed by a collaboration between Natural 
and Artificial Cognition Lab (NAC) of University of Naples Federico II and INDIRE, the National 
Institute for Documentation, Innovation and Educational Research. 

A special and heartful thanks goes to the memory of Prof. Orazio Miglino, whose ideas, 
efforts, and intuitions dramatically impacted on the realization of this study. 


References 


1. Hirst, M., Jervis, N., Visagie, K., Sojo, V., Cavanagh, S.: Transition to primary school: a 
review of the literature. Commonwealth of Australia, Canberra (2011) 


102 F. Diano et al. 


2. Lewit, E., Baker, L.: School readiness. Future Child. 5(2), 128-139 (1995). https://doi.org/ 
10.2307/1602361 

3. Kienig, A.: The importance of social adjustment for future. In: Transitions in the Early Years: 
Debating Continuity and Progression for Young Children in Early Education, pp. 23-37 
(2002) 

4. Doherty, G.: Conception to Age Six: The foundation of School-Readiness. Learning 
Partnership, Toronto (2007) 

5. Italian Ministry for Education University and Research: Indicazioni nazionali per il curricolo 
della scuola dell’infanzia e del primo ciclo d’istruzione (2012) 

6. Diano, F., Ferrara, F., Rosa, A., Ponticorvo, M.: Diligo 2.0: a digital assessment tool for 
geometric and emotional skills in preschoolers. In teleXbe (2021) 

7. Kahneman, D.: Thinking, Fast & Slow. F. Straus and Giroux, New York (2011) 

8. Miglino, O., Di Ferdinando, A., Schembri, M., Caretti, M., Rega, A., Ricci, C.: STELT (smart 
technologies to enhance learning and teaching): una piattaforma per realizzare ambienti di 
realtà aumentata per apprendere, insegnare e giocare. Sistemi Intell. 25(2), 397-404 (2013) 

9. Helbing, D.: Agent-based modeling. In: Helbing, D. (eds.) Social self-organization, pp. 25-70. 
Springer, Berlin, Heidelberg (2012) https://doi.org/10.1007/978-3-642-24004-1_2 

10. Miglino, O.: Festina lente. Apprendere ad armonizzare i processi di pensiero lento e veloce 
come precursore dell’imparare a ragionare [Learning to harmonize slow and fast thinking as a 
precursor of learning how to reason]. G. Ital. Psicol. [Ital. J. Psychol.] 46(4), 859-864 (2019) 

11. Bruner, J.: Play, thought and language. Prospects 16(1), 77-83 (1986) 

12. Casey, B., Erkut, S., Ceder, I., Young, J.M.: Use of a storytelling context to improve girls’ 
and boys’ geometry skills in kindergarten. J. Appl. Dev. Psychol. 29(1), 29-48 (2008) 

13. Smorti, A.: II pensiero narrativo: costruzione di storie e sviluppo della conoscenza sociale 
[The narrative thinking: building stories and social cognition developemnt], Giunti, Firenze 
(1994) 

14. DILIGO 2.0 Google Play Store page. https://play.google.com/store/apps/details?id=com.nac 
playground.diligo2&hl=en&gl=US 

15. DILIGO 2.0 Abilita Socio-Emotive Google Play Store page. https://play.google.com/store/ 
apps/details?id=com.nacplayground.diligo2ase&hl=en&gl=US 


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 
International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, 
adaptation, distribution and reproduction in any medium or format, as long as you give appropriate 
credit to the original author(s) and the source, provide a link to the Creative Commons license and 
indicate if changes were made. 

The images or other third party material in this chapter are included in the chapter’s Creative 
Commons license, unless indicated otherwise in a credit line to the material. If material is not 
included in the chapter’s Creative Commons license and your intended use is not permitted by 
statutory regulation or exceeds the permitted use, you will need to obtain permission directly from 
the copyright holder. 


D 


Check for 
updates 


Robot Assistive Therapy Strategies for Children 
with Autism 


Grazia D’Onofrio!, Annamaria Petito? œ, Antonella Calvio?, Giusi Antonia Toto}, 
and Pierpaolo Limone? 


! Clinical Psychology Service, Health Department, Fondazione IRCCS Casa Sollievo Della 
Sofferenza, San Giovanni Rotondo, Foggia, Italy 
e Department of Clinical and Experimental Medicine, University of Foggia, Foggia, Italy 
annamaria.petito@unifg.it 
E Department of Humanistic Studies, University of Foggia, Foggia, Italy 


Abstract. Background: Autism spectrum disorder (ASD) is a category of neu- 
rodevelopmental disorder characterized by persistent deficits in social communica- 
tion and social interaction across multiple contexts as well as restricted, repetitive 
patterns of behaviour, interests, or activities. Social robots offer clinicians new 
ways to interact and work with people with ASD. Robot-Assisted Training (RAT) 
is a growing body of research in HRI, which studies how robots can assist and 
enhance human skills during a task-centred interaction. RAT systems have a wide 
range of application for children with ASD. 

Aims: Ina pilot RCT with an experimental group and a control group, research 
aims will be: to assess group differences in repetitive and maladaptive behaviours 
(RMBs), affective states and performance tasks across sessions and within each 
group; to assess the perception of family relationships between two groups before 
and post robot interaction; to develop a robotic app capable to run Raven’s Progres- 
sive Matrices (RPM), a test typically used to measure general human intelligence 
and to compare the accuracy of the robot to capture the data with that run by 
psychologists. 

Material and Methods: Patients with mild or moderate level of ASD will be 
enrolled in the study which will last 3 years. The sample size is: 60 patients (30 
patients will be located in the experimental group and 30 patients will be located 
in the control group) indicated by an evaluation of the estimated enrolment time. 
Inclusion criteria will be the following: eligibility of children confirmed using the 
Autism Diagnostic Observation Schedule —2; age > 7 years; clinician judgment 
during a clinical psychology evaluation; written parental consent approved by the 
local ethical committee. The study will be conducted over 10 weeks for each par- 
ticipant, with the pretest and post test conducted during the first and last weeks of 
the study. The training will be provided over the intermediate eight weeks, with 
one session provided each week, for a total of 8 sessions. Baseline and follow-up 
evaluation include: socioeconomic status of families will be assessed using the 
Hollingshead scale; Social Communication Questionnaire (SCQ) will be used to 
screen the communication skills and social functioning in children with ASD; 
Vineland Adaptive Behavior Scale, 2nd edition (VABS) will be used to assess 
the capabilities of children in dealing with everyday life; severity and variety of 
children’s ripetitive behaviours will be also assessed using Repetitive Behavior 
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Scale-Revised (RBS-R). Moreover, the perception of family relationships assess- 
ment will be run by Portfolio for the validation of parental acceptance and refusal 
(PARENTS). 

Expected Results: 1) improbe communication skills; 2) reduced repetitive 
and maladaptive behaviors; 3) more positive perception of family relationships; 
4) improved performance. 

Conclusions: Robot-Assisted Training aims to train and enhance user (phys- 
ical or cognitive) skills, through the interaction, and not assist users to complete 
a task thus a target is to enhance user performance by providing personalized and 
targeted assistance towards maximizing training and learning effects. Robotics 
systems can be used to manage therapy sessions, gather and analyse data and like 
interactions with the patient and generate useful information in the form of reports 
and graphs, thus are a powerful tool for the therapist to check patient’s progress 
and facilitate diagnosis. 


Keywords: Psychology - Learning - Technology - Autism - Robot Assisted 
Training 


1 Introduction 


Autism spectrum disorder (ASD) is a category of neurodevelopmental disorder char- 
acterized by persistent deficits in social communication and social interaction across 
multiple contexts as well as restricted, repetitive patterns of behaviour, interests, or 
activities [1]. The care and social needs of preschool children with ASD (typically up to 
six years of age), in particular, are significant [2, 3], usually extend to parents and siblings 
[2, 4, 5], and require substantial community resources [2, 6, 7]. Estimated costs per year 
for children affected by Autistic Spectrum Disorders (ASD) in the US are expected to be 
between $ 11.5 billion — $ 60.9 billion (2011 US $), representing a significant economic 
burden from medical care to private education [8, 9]. An increasing amount of research 
has investigated the use of digital interventions for support and treatment of ASD individ- 
uals under terms such as digital health [10], computer-based [11, 12], computer-assisted 
[13], innovative technology-based [14] and technology-aided interventions [15]. Devices 
like computers, smartphones, wearable technologies, virtual reality, robotics and tablets 
[13] have been used. The interventions attempt to teach or train e.g. communication [9], 
social and emotional skills [10] and academic skills [11]. The use of digital interven- 
tions may be beneficial for ASD individuals, since such interventions are consistent, 
predictable, and without social interaction, which is preferred by ASD individuals [16]. 
Computer-assisted and robot-assisted therapy is infiltrating the social skills teaching 
environment, being trailed or incorporated into therapy by a variety of professions to 
help teach the child with ASD [17-19]. Validation of the effectiveness of computer-aided 
therapies to teach social skills is warranted to justify the quality of these interventions. 
Useful technologies will likely proliferate further into therapy regimens, offering new 
models and assistance to those who serve these children and their families. Animal-like 
robots and humanoid robot have received especially notable acceptance in therapeutic 
settings. The notion that children with ASD prefer robots as tutors to improve their 
social interaction and communication abilities is supported by recent studies. Indeed, 
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the research focused on developing a very promising form of intervention called robot- 
assisted therapy. This therapy has some challenges, e.g., the necessary flexibility and 
adaptability to real unrestricted therapeutic settings. The most frequent deficiency to 
children with autism and mental disability is social attention, which includes the dif- 
ficulty of focusing good visual attention. Di Nuovo and his colleagues [20] examined 
the use of a new deep learning neural network architectures to automatically determine 
whether a child-focused on visual attention during a therapeutic session, indicating their 
commitment. They used the NAO humanoid robot [21] for their research and have pro- 
posed the use of computational intelligence techniques to increase robot capabilities for 
greater adaptability and flexibility, enabling the robot to be integrated into any therapeu- 
tic environment, according to the specific needs of the therapist and the individual child. 
A study by Huskens et coll. Utilized a robot-mediated intervention based on LEGO® 
Therapy to study the impact on collaborative play behaviour [22, 23], using NAO robot. 
The robot reinforced collaboration and offered prompts. 

In another study by Srinivasan et al. [24] utilizing the rhythm and robotic therapy 
intervention with NAO and Rovio, the outcome targets were repetitive behaviours and 
affective states in children with ASD. After training, the rhythm group reduced negative 
behaviours. Affective state results indicated the rhythm and robotic groups demonstrated 
greater interested affection across all sessions. Negative affection was decreased and 
interested affection increased in the rhythm group after training. Other studies have 
reported improvement in communication, social skills and gestural delay for a child 
with ASD after robot therapy intervention [25-31]. These studies highlight the uses of 
robot-assisted interventions to teach social skills to children with ASD. The systematic 
review by Grossard and colleagues [32] reported excellent state of the art in the topic 
ICT and autism care from 2017 to 2018. They analyzed serious games and social robots. 
The authors noted children with ASD have a specific need for predictability, visual 
support, and a sequential presentation of information, which aligns well with the use of 
social robots. They concluded that social robots offer clinicians new ways to interact 
and work with people with ASD. Robot-Assisted Training (RAT) is a growing body of 
research in HRI, which studies how robots can assist and enhance human skills during 
a task-centred interaction. RAT systems have a wide range of application from physical 
assistance in post-stroke rehabilitation and robotic prosthetics [33], to cognitive training 
for patients suffering from dementia, Alzheimer’s disease [34, 35], MCI and intervention 
and therapy for children with ASD. Socially Assistive Robotics (SAR) is also employed 
for language learning and children education [36, 37]. HRI is a multidisciplinary research 
field that involves human-machine interaction, machine learning, data mining, computer 
vision as well as psychology and educational sciences, kinesiology, occupational therapy 
and others. The main difference from other assistive robotic systems is that Robot- 
Assisted Training aims to train and enhance user (physical or cognitive) skills, through 
the interaction, and not assist users to complete a task (Activities of Daily Living) thus a 
target is to enhance user performance by providing personalized and targeted assistance 
towards maximizing training and learning effects. Applied behaviour analysis (ABA) is 
one of the most extended therapies for the treatment of autism, consisting of improving 
specific behaviours which are divided into simple and repetitive tasks that are presented 
sequentially and strategically while measuring and analysing the patient’s performance 
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during the therapy [38]. Robotics systems can be used to manage therapy sessions, gather 
and analyse data and like interactions with the patient and generate useful information 
in the form of reports and graphs, thus are a powerful tool for the therapist to check 
patient’s progress and facilitate diagnosis. The visual appeal of the robotics platform is 
a key factor to engaging the attention of children with autism; indeed those robots tend 
to use bright colours, rotating mechanical parts, striking shapes and lights [39]. 


2 Objective 


In a pilot RCT with an experimental group and a control group, research aims will be: 


to assess group differences in repetitive and maladaptive behaviours (RMBs), affective 
states and performance tasks across sessions and within each group; 

to assess the perception of family relationships between two groups before and post 
robot interaction; 

to develop a robotic app capable to run Raven’s Progressive Matrices (RPM), a test 
typically used to measure general human intelligence and to compare the accuracy of 
the robot to capture the data with that run by psychologists. 


3 Application Field 


Patients with mild or moderate level of ASD to enrol in Fondazione IRCCS Casa Sollievo 
della Sofferenza, San Giovanni Rotondo (FG). 

Duration of the study: 3 years. 

Sample size: 60 patients (30 patients will be located in the experimental group and 30 
patients will be located in the control group) indicated by an evaluation of the estimated 
enrolment time. 

Inclusion criteria will be the following: 


1) eligibility of children confirmed using the Autism Diagnostic Observation Schedule 
—2, a gold standard diagnostic assessment for ASD [40]; 

2) age > 7 years; 

3) clinician judgment during a clinical psychology evaluation; 

4) written parental consent approved by the local ethical committee. 


4 Description of the Process Phases, Indicators and Monitoring 


4.1 Procedure 


The study will be conducted over 10 weeks for each participant, with the pretest and 
posttest conducted during the first and last weeks of the study. The training will be 
provided over the intermediate eight weeks, with one session provided each week, for a 
total of 8 sessions. 
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4.2 Baseline and Follow-up Evaluation 


The socioeconomic status of families will be assessed using the Hollingshead scale [41]. 
The Social Communication Questionnaire (SCQ) [42] was used to screen the communi- 
cation skills and social functioning in children with ASD. Vineland Adaptive Behavior 
Scale, 2nd edition (VABS) [43] will be used to assess the capabilities of children in deal- 
ing with everyday life (i.e., communication skills, motor skills, functionalities needed in 
everyday life, and socialization). It will be also assessed the severity and variety of chil- 
dren’s repetitive behaviours using the parent questionnaire, Repetitive Behavior Scale- 
Revised (RBS-R) [44]. Moreover, the perception of family relationships assessment will 
be run by Portfolio for the validation of parental acceptance and refusal (PARENTS) [45]. 


4.3 Robot-Mediated Training 


Among android social assistive robots, Pepper from Softbank Robotics is a humanoid 
(height = 1.21 m and width = 0.48 m) that has almost the same articulations as a human, 
except for its mobile base and the impossibility of moving every finger independently. 
It has four microphones, two loudspeakers, two RGB cameras and a depth sensor (Asus 
Xtion). It has tactile sensors in the head and the back of its hands. It has a speech recogni- 
tion engine that is able of identifying multiple variations in the human voice and a speech 
to text module that makes it talk with people, as well as face detection and recognition 
and emotion detection engines. It is also provided with a tablet, enabling it in showing 
interactive content. Nao, Pepper’s little brother with similar software characteristics, is 
also a humanoid (height = 0.57 m and weight = 5 kg) appearing as of a human toddler 
and already widely adopted in ABA with ASD. Pepper and Nao are equipped with cam- 
eras and software for eye-tracking, face detection and recognition however it will not be 
employed in this study (Fig. 1). 


\ 


pepper 


Fig. 1. Pepper and Nao robots from Softbank Robotics. 
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Robot-assisted therapy today is available through applications running on the robot 
Pepper itself; the Research Hospital IRCCS “Casa Sollievo della Sofferenza” is licensed 
with a solution named Robomate, developed by BehaviorLabs. Robomate is an LMS 
(Learning Management System) platform for humanoid robots, created for the following 
purposes (Fig. 2): 


Simplify the use of robots by clinicians, therapists and educators 

Realize an easy and intuitive platform for the human-machine interaction 
Handle e-learning contents and “edutainment” 

Manage contents 

Track and store results of the executed sessions and patient data 
Generate reports and statistics on the results of the executed sessions 
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Fig. 2. Managing users’ sessions and results on Robomate. 


Robomate is then tailored for the use of a humanoid robot, representing a direct 
communication channel between the content and the child, and a tablet device as a 
reinforcing and feedback element where the child himself can answer and interact in 
the session. In this sense, the robot and the tablet work together both as executors and 
catchers of feedbacks during the training sessions. Through the iPad app Robomate, a 
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therapist can register patient data and sessions as well as trigger predefined animations 
on the robot to catch the attention during a test session. Telepresence on the app through 
Pepper's mics and cameras is also available (Fig. 3). 


Fig. 3. Robomate app for remote controlling Pepper. 


Following, a description of the exercises available on the robot through Robomate. 
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4.4 Interactive Sessions with the Robot 


Interactive sessions of the child with the robot will take place after the baseline session 
and before the follow-up evaluation session. Three standard session types have been 
designed with a predefined set of tests selected from the table above that the therapist 
will administer using the robot (Table 1 and 2): 


Table 1. Set of available exercises on Pepper through Robomate. 


Game name Learning objective Description 


Riconosci gli animali Identification Pepper asks to recognize animals from 
paints on paper and to show them to its 
camera for acknowledgement 


Gioco delle imitazioni Identification Pepper mimics objects and animals and 
asks to identify them 

Caravan Palace Dance Dance-Animation Pepper dances on the notes of the song 
“Caravan Palace” 

Thriller Dance Dance-Animation Pepper dances on the notes of the song 
“Thriller”, by Michael Jackson 

Around the World Dance-Animation Pepper dances on the notes of the song 
“Around the World”, by Daft Punk 

Row Row Row Your Boat | Dance-Animation Pepper dances on the notes of the song 
“Row Row Row Your Boat” 

Ai Se Eu Te Pego! Dance-Animation Pepper dances on the notes of the song 
“Ai Se Eu Te Pego!” 

Come mi sento? Emotions Pepper tells stories and asks to recognize 


emotions felt by characters 


Training imitazione A Physical Imitation Imitation training of non-significative 
movements of one limb 


Training imitazione B Physical Imitation Imitation training of non-significative 
movements of two limbs 


Training imitazione C Physical Imitation Imitation training of non-significative 
movements of two hands 

Training imitazione D Physical Imitation Imitation training of 10 movements from 
A, B, C repeated 4 times 

Training imitazione F Physical Imitation Imitation training of crossed movements 

Training imitazione H Physical Imitation Two different answers at the time 

Training imitazione I Physical Imitation 20 movements, Pantomime 

Test Geometria Object Identification Pepper asks to identify geometric shapes 

Prerequisiti Visivo Identification Visual perception, identification of 
sequences and spatial position of the 
objects 


(continued) 
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Table 1. (continued) 


Game name Learning objective Description 

Lettura Vocali (Test) Identification Pepper asks to identify the right answer 
for the presented vowel 

Lettura e Discriminazione | Identification Pepper asks to identify the group of 
vowels for a prompt 

La colomba e la formica 1 | Identification Demonstrate a good auditory memory, 
grasping the details of an overheard story 

Matematica Posizioni Identification Pepper asks about the location of the 
objects 

Digit Span Test Efficiency and Attention | WAIS and WMS sub-test where Pepper 


dictates sequences of digits and the 
patient should repeat them in the same 
order 


Token Test Oral comprehension Pepper shows on its tablet a set of 
coloured geometrical shapes and asks 
the interviewee to touch or move some 
of them, to evaluate the residual 
potential of the understanding of the oral 
language 


Table 2. Set of exercises for each session type. 


Session Type 1 Session Type 2 Session Type 3 
Orientamento meteorologico | Orientamento meteorologico | Orientamento meteorologico 
Come mi sento? Riconosci gli animali Token Test 

Training imitazione mix Training imitazione mix Come mi sento? 

Prerequisiti Visivo Come mi sento? Training imitazione mix 
Canzone per ballare Canzone per ballare Canzone per ballare 


In the eight interactive sessions with Pepper, the schemas proposed in the table above 
will be administered as follows in Fig. 4. 
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Fig. 4. Timeline of the sessions. 


5 Ethical Considerations 


5.1 General Ethical Aspects in the Conduction of the Study 


Declaration of Helsinki. 

The current version of the Declaration of Helsinki (2013) is a reference for the ethical 
aspects of this clinical study and it will be respected by those employed in this research 
study. 


5.2 Ethical Committee 


The Principal Investigator of the study will present to the Ethical Committee of the 
Research Hospital IRCCS “Casa Sollievo della Sofferenza” in San Giovanni Rotondo. 
The study protocol including documentation relating to the information to be provided 
to patients and forms for obtaining informed consent to medical treatment, informed 
consent to the processing of personal data and any other document necessary for them 
to fulfil their responsibilities. The study will be started only after obtaining the written 
approval, dated and signed by the Ethical Committee, under the guidelines of Good 
Clinical Practice (Buona pratica Clinica, Italian Ministerial Decree No. 162 of July 15, 
1997) and the Italian Ministerial Decree of March 18, 1998 “Linee Guida per l’istituzione 
ed il funzionanamento dei Comitati Etici” published in the Official Gazette No. 122 of 
May 28, 1998. 


5.3 The Informed Consent Form and Information Sheet 


Before carrying out any study procedure on a patient belonging to the centre, the Inves- 
tigator must obtain the acquisition of all written consent. It is the Investigator’s responsi- 
bility to provide comprehensive information relating to the study’s operating procedures. 
The documents that identify the subject will be kept confidential and, under EU Reg- 
ulation 2016/679 (GDPR) and Legislative Decree 196/2003, will not be made publicly 
available or communicated to unauthorized subjects. If the results of the study are pub- 
lished, the identity of the subject will remain secret. The patient must be informed that 
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participation in the study is voluntary and that refusal to participate does not imply any 
penalty, that he has the right to terminate his participation in the study at any time and that 
this decision will not affect future care. Before the start of the study, the Investigator will 
deliver to each patient’s parent the Information Sheet, containing the description of the 
study and the telephone number of the Investigator. The subject will be given sufficient 
time to be able to decide on possible participation in the study and to obtain any necessary 
clarification from the Investigator regarding the study. Furthermore, the patient’s parent 
must carefully read, and understand, what is present in the “Prospect - Informed Con- 
sent” form, section “Information on Data Processing”; subsequently, he/she may affix 
his written consent. If the patient’s parent is in favour of participating in the study, he/she 
will personally sign and date the “Prospect - Informed Consent” form. The Investigator 
will in turn sign the Informed Consent form. A copy of the aforementioned forms, duly 
signed, will be delivered to the patient’s parent. 


5.4 Data Recording and Storage 


The data collected through electronic devices and questionnaires will report the codes 
assigned to patients. The codes will be designed in such a way as not to allow inferring the 
identity of the subject to which they refer. All documents that could disclose the identity 
of patients will be kept by the Investigator in strict confidentiality, under EU Regula- 
tion 2016/679 (GDPR) and Legislative Decree 196/2003. All documents containing the 
original data and transcribed in the CRF will remain in the archives of the Unit of Child 
Neuropsychiatry of the Research Hospital IRCCS “Casa Sollievo della Sofferenza”. The 
signed originals of the forms, the hospital documentation and other documents relating 
to the study must be archived by the Investigator and kept in the limited access archives 
of the Unit of Child Neuropsychiatry “Casa Sollievo della Sofferenza”. No documents 
relating to the study may be destroyed without prior authorization from the investigators. 


5.5 Financial Aspects 


This kind of research will not entail any cost for the Italian National Health Service. 
As this is a spontaneous study, the participating investigators will not receive any 
compensation by way of reimbursement for the research activities carried out. 


6 Conclusions 


Robot-Assisted Training (RAT) aims to train and improve both physical and cognitive 
skills of the user, through interaction, so one of the goals of this study is to improve user 
performance by providing personalized and targeted assistance towards maximizing 
training and learning effects. Robotic systems can be used to manage therapy sessions, 
collect and analyze both data and parental interactions and perceptions of family rela- 
tionships. One of the most disabling deficits of children with ASD is difficulty in commu- 
nication and social interaction. Specifically, children with ASD may exhibit difficulties 
with social-emotional reciprocity; deficits in nonverbal communicative behaviors used 
for social interaction; and deficits in the development, management, and understanding 
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of relationships. RAT appears to be a valuable support for improving communication 
skills. In addition, the study aims to reduce repetitive and maladaptive behaviors through 
the use of specific therapy sessions that will help children with ASD to manage repet- 
itive and maladaptive patterns of behavior, interest or activities. The RAT supports the 
processing of useful information in the form of reports and graphs, proving to be a 
powerful tool for the therapist in order to monitor the patient’s therapeutic progress and 
facilitate in a timely manner the formulation of a correct diagnosis, useful also for the 
multidisciplinary team (educational, psychological, medical). 
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Abstract. Education digital storytelling (EDS) refers to digital story- 
telling in education. EDS is often presented as a technology-enhanced 
learning approach with powerful educational benefits. The use of EDS 
as a tool to support students to consider and pursue a meaningful life 
based on their interests, talents, and aspirations is spreading. Due to its 
nature, EDS could be an optimal tool to create and share digital stories 
to stimulate career plans. This paper deals with the educational uses of 
digital storytelling and presents an overview of how EDS can be used 
to support career guidance. It concludes with a brief description of the 
NEFELE Erasmus+ project that will implement tangible user interfaces 
(TUIs) in the classrooms to promote the EDS as a tool for career inter- 
vention. 
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design 


1 Introduction 


Educational digital storytelling (EDS) has grown out of Digital Storytelling (DS) 
in the past two decades. EDS refers to digital storytelling in the field of education, 
such as schools. 

EDS is often presented as a technology-enhanced learning approach with 
powerful educational benefits. Starting from Lambert’s definition of DS as a 
short narrated film [1], is it possible to define EDS as facilitated production of 
a short digital story in an educational community setting that should contain 
a mixture of digital media, including text, recorded audio narration, pictures, 
music and video. 

Due to its nature, EDS could be an optimal tool to create and share digital 
stories to stimulate career plans. Students can think about their futures and use 
media to express their aspirations, expectations, needs and dreams in a narrative 
way. Although often used as a tool for agentic identity development [2,3], its use 
as a career guidance tool is very recent [4]. 
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This paper deals with the educational uses of digital storytelling and presents 
an overview of how EDS can be used to support career guidance. The first section 
“Education Digital Storytelling” begins with a definition of EDS and focuses on 
how EDS is being used in education as support teaching and learning. The second 
part “Life design paradigm” proposes career guidance interventions focusing on 
the life design (LD) paradigm. The following part “Link between Educational 
Digital Storytelling and Life Design” discusses the use of EDS as a tool for LD 
career intervention. The paper concludes with a description of the NEFELE 
ERASMUS+ project that will implement tangible user interfaces in classrooms 
to promote the use of EDS as a tool for LD career intervention. 


2 Educational Digital Storytelling 


“When we enter human life, it is as if we walk on stage into a play whose 
enactment is already in progress — a play whose somewhat open plot determines 
what parts we may play and toward what denouements we may be heading” ( 
[5], p.34). 

Storytelling is sharing ideas, needs, and experiences through words to make 
meaning about our lives [5]. Through narrative, individuals shape their imag- 
ination and reflect on the concept of “the possible”. In this sense, the focus 
is transferring meaning [6]. Storytelling practice enhances the ability to nego- 
tiate new meanings and understandings and allows bringing together both the 
canonical and the exceptional and cultivating the possible [5]. 

Digital Storytelling (DS) offers new instruments for revisiting storytelling, 
blending multimedia, interactivity and the web into traditional storytelling prac- 
tices [7,8]. It can be defined as a modern expression of ancient storytelling [8]. 
Individuals become active creators by creating electronic personal narratives 
[9-11].DS is conceived as a new creative cultural mode [7]. 

Seven elements of DS are presented by Lambert [9,12]: point of view, a dra- 
matic question, emotional content, the gift of your voice, the power of the sound- 
track, economy, pacing. Point of view refers to the personal perspective of the 
narrator and the sense of authenticity of the story. A dramatic question is defined 
as an “existential” question to which the narrator wants to find an answer at 
the end of the story. Emotional content reflects the emotional connection to the 
story. The gift of voice and the power of the soundtrack refers to the personal- 
ization of the story. Finally, economy and pacing refer to just enough content to 
tell the story and its pacing (slowly or quickly). 

Robin [13] describe the types of digital stories: personal narrative, which 
includes stories with significant incidents in the narrator’s life; historical docu- 
mentaries, which refers to the exploration of dramatic events in search of mean- 
ing; stories designed to train concepts or practices. 

DS was applied in different fields, e.g. health [14] and youth civic engagement 
[15]. In the last year, its application in education has grown [16]. DS storytelling 
offers a meaningful context for the effective integration of digital technologies in 
the classroom [17]. 
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In a recent review, Wu and Lee [16] showed five types of EDS orientation: 
appropriative, agentive, reflective, reconstructive, and reflexive. The appropria- 
tive EDS is designed for storytellers to understand through production. The 
agentive EDS is structured to enhance autonomy. The reflective EDS is designed 
to engage in a reflective process. The reconstructive EDS refers to the process 
of reconstructing meanings of a concept. Finally, the reflexive EDS allows to 
negotiate and manage the understanding of self and others. 

Eight types of outcomes of EDS can be identified [16]: affective, cognitive, 
conceptual, academic, technological, linguistic, ontological, and social. Affective 
refer to learner attitude (e.g., motivation, empathy). Cognitive outcomes refer 
to creative thinking. Conceptual outcomes are concerned with a reformulation 
of concepts. Academic, technological and linguistic outcomes refer to the related 
skills. Ontological skills refer to identity topics. Finally, social outcomes are con- 
cerned with all the collaborative skills. Moreover, learning to construct and tell 
stories is a means of defining and reflecting on personal experiences and com- 
municating them to others. 

In the context of disability, structuring storytelling educational interventions 
is challenging. New strategies and forms of storytelling practices are mainly 
designed for typically developing pupils. However, there is no lack of evidence 
of the potential of storytelling and DS tools in special and inclusive education 
contexts for students with disabilities or special educational needs (SEN) such as 
physical and sensory disability, learning disorders or intellectual disability [18]. 

Particularly, DS activities could be useful and engaging to support the expres- 
sive abilities of children and young people with disabilities or SEN. One of the 
toughest difficulties of pupils with disabilities and SEN, indeed, concerns the 
ability to communicate. Communication impairments range from language dis- 
orders, inability to effectively implement non-verbal communication or difficulty 
expressing feelings and thoughts; they all generate negative emotions and inter- 
fere with learning and development (DSM-5, [19]). 

Neurodevelopmental disorders, such as intellectual disorders or autism spec- 
trum disorders, often pervasively affect children’s language learning and com- 
munication abilities; then, one of the main interventions’ purposes is to teach 
them effective expression strategies. In order to reach this goal, visual communi- 
cation tools for Augmented Alternative Communication (ACC), such as PECS 
protocol (Picture Exchange Communication System), were developed, but also 
new technologies that empowered this communication system based on a solid 
Applied Behavior Analysis (ABA) methodological framework [20]. In one study 
[21], researchers integrated the PECS communication system with a digital story- 
telling intervention for disabled children who suffer severe impairments in body 
and fine-grained movements and limited cognitive functions such as memory, 
problem-solving, attention, visual and language comprehension. The approach 
was applied in a real school context, where the benefits for disabled children 
were evident in positive emotional expression, linguistic and narrative capabil- 
ity improvement, moreover in autonomy and movement control, but also for 
classmates in social cohesion and a stronger sense of community. 
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One aspect that makes DS particularly suitable for pupils with disabilities 
or SEN is that it is a highly adaptable and flexible methodology; therefore, it 
allows the personalization of interventions and features of the tools used based 
on the needs, potential and competences of individuals. Having identified spe- 
cific objectives, teachers and educators can organize storytelling activities and 
materials (visual, auditory, and more) so that each student can make the most of 
their personal resources. DS is also a cooperative learning tool, as it stimulates 
children with SEN to interact and collaborate with their peers to develop social 
and interpersonal skills, thus promoting true inclusion. 

Certainly, to fully exploit the support of digital media for storytelling and 
involve vulnerable children in the development of expressive skills, it is important 
to customize the design and intervention based on the differences of each one. 
Botturi and colleagues [18] developed a model, the digital storytelling for devel- 
opment model (DSD), to support DS activities implementation in educational 
contexts for children with SEN. 


3 Life Design Paradigm 


Career transitions represent critical developmental tasks in adolescence and early 
adulthood [22]. In the present era, a new social arrangement of work poses a series 
of concerns and challenges in careers [23,24]. 

Due to this unstable environment, the relationship between adolescents and 
career choices must be carefully rethought [25]. New barriers interrupt the linear- 
ity of career transitions, and adolescents have to cope with unpredictable career 
trajectories increasingly. Moreover, the career choices do not end at a given 
moment in life (i.e., school-to-work transition), but the number of career tran- 
sitions that individuals must make in their lifetime has dramatically increased 
[26]. 

The transitions among educational systems are the crucial career decision- 
making moments [27,28]. During these transitions, the quality of career decisions 
and the difficulties that the individual encounters play a central role. Transitions 
like this can promote indecision [29] and are defined as the most widespread 
vocational problem [30]. 

Furthermore, adolescents are currently thinking about their future in this 
continuously changing context. These environmental risk factors could inhibit 
the development of the future orientation in adolescents [31], leading to a lack 
of optimism and hope toward themselves and their future [31]. The negative 
feelings about their future possibilities could become obstacles to adolescents’ 
career development and restrict their future expectations [32]. 

Becoming prepared for a career is necessary to master career transitions [33]. 
Lifelong learning, counseling and life designing are needed to help individuals 
access sustainable careers [34]. In this sense, constructing a sustainable career 
requires a preventive perspective in guidance and career intervention. 

Interventions should guarantee a positive orientation toward future vision 
promoting hope and optimism [35] and enhancing coping strategies. 
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Although traditionally career guidance followed the person-environment fit 
approach that aimed to assess people's skills and interests for oriented the suit- 
able job [36], from the end of the 20th century the Life Design (LD, [37,38]) 
was the leading paradigm. The LD model for career intervention endorses that 
preventive career interventions with children and adolescents are helpful to pre- 
pare adolescents for their transition, increase their choice opportunities and 
decrease at-risk situations and social inequalities [37]. LD conceives career as 
a self-directed process: the adolescents through personal negotiation meaning 
in social interactions shape their life course. In this paradigm, adolescents are 
actively engaged in the career development process and enhance the needed skills 
that can lead them to successfully interact with their environment [24]. In the 
LD model, individuals are required to tell small stories. Storytelling allows to 
make the self and crystallizes what individuals think of themselves. Savickas 
argues that “the more stories they tell, the more clients develop their identities 
and careers” ([37], p.15). LD allows individuals with and without disabilities to 
aspire to an inclusive and sustainable future [39]. 

Students with disabilities and SEN, due to lack of targeted opportunities 
for reflection, expression and support in career and life choices, often develop 
feelings of anxiety and worry, motivational and cognitive beliefs that include 
a low sense of self-efficacy [40]. It is important to emphasize that people in 
disadvantaged conditions, such as disability, can become more self-determined, 
autonomous, satisfied with their life and able to make life choices if they are 
provided with adequate support [41]: it is, therefore, essential to create self- 
expression spaces to stimulate reflection and awareness on personal resources, 
desires, and expectations for their future life. Therefore, the storytelling and 
narrative approach are not limited to the creation of educational spaces for 
linguistic-narrative development: they are tools to increase the self-awareness 
of people with disabilities that aim to design a life path as autonomous and 
independent as possible. 

To promote disabled persons career guidance, it is necessary to provide more 
opportunities for reflection and expression. Young people with disabilities, in an 
inclusive perspective, should be helped to recognize and focus on their resources 
and strengths and the skills and competences needed to identify life goals. In 
addition, they could be stimulated to talk about themselves in a positive way, 
as well as their peers, to support positive visions of themselves and the future, 
helping them to overcome the stigmatization of their disability, obstacles, and 
barriers to their positive personal and professional development [40]. 


4 Link Between Educational Digital Storytelling and Life 
Design 


In an EDS, students are encouraged to express their life stories jointly using 
music, video, and recorded voice. In the life design paradigm, individuals are 
encouraged to write their life stories. Career stories are at the heart of story- 
telling. Storytelling has a transformative role in enhancing adolescents’ ability 
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in making a clear statement of their identity and helps them express their future 
expectations. LD paradigm believes that adolescents give meaning to careers 
through stories creating a narrative identity [38]. 

Pordelan and colleagues [4] have integrated the use of DS in LD intervention. 
The authors refer to specific career counseling sessions, and the DS was used 
as one part of the LD protocol. The findings showed that the use of digital 
storytelling in the LD promotes students’ career decision-making. 

No studies have proposed the use of DS in a school setting, defined as EDS, 
to support career guidance. 

We believe that the EDS can be used as a tool to support students to consider 
and pursue a meaningful life based on their interests, talents and aspirations. 
Telling is a crucial ingredient of peoples’ attempts to construct, deconstruct, 
reconstruct, and co-construct their career-life stories [42]. Due to the nature of 
the reflective practice [43], EDS can be a tool for telling career stories. In this 
way, EDS may help students talk about their needs easily and relate their present 
as students and future and life trajectories [44,45] and their agency to pursue 
their aspirations [45]. 

EDS is a flexible and adaptable tool. Consistent with Pordeland and col- 
leagues [4], the novelty approach of digital storytelling in LD is that over the 
development process because the story can be modified, enhancing career self- 
efficacy. Through “career digital storytelling”, adolescents can express their own 
career identity in terms of life themes [37,38] and the identifications of a career 
path. This capability allows them to face social, contextual, and career changes, 
interpreting them as opportunities. 

Moreover, as an innovative and flexible tool, EDS could empower the orien- 
tation and career guidance process of students with SEN: EDS opens to different 
and creative ways of expression. The digital features of EDS, such as images, 
music, different presentation forms, allow a selection of those techniques that 
best suit each student. To view and show one story means narrating differently 
than through words. It means being able to listen to it several times and modify 
it, if necessary, in the process of continuous reflection and construction of oneself 
and one’s life story. Storytelling with images means understanding everything 
from a new point of view, and if it is a good exercise for all, vulnerable children 
could discover new strategies for naming emotions and ideas. 


5 NEFELE Erasmus+ Project 


Against this background, the NEFELE Erasmus+ project aims to develop an 
authoring tool that allows the creation of games for middle school pupils to 
support career choices. 

This innovative tool allows students to experiment with themselves in differ- 
ent roles, opening to a positive future vision and enhancing planning and future 
orientation skills. The game, in this sense, is an elective tool for engaging learn- 
ing, in which children in an authoring system will be able to construct their 
contents. 
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Alongside the tangibility of the real objects and multisensorial experience, the 
storytelling tool itself will allow adolescents to think about the future, encour- 
aging meaning-making processes and enhancing a positive vision of the future 
characterized by hope and optimism. 

Consistent with Maree [42], the school provides the adolescent with a much- 
needed holding environment. The ability to hold refers to enabling adolescents 
to make meaning in their careers. In line with this point of view, teachers in the 
school environment should allow adolescents to acknowledge and validate their 
narratives [46]. 

The authoring tool will be based on the Tangible User Interface paradigm. 
Tangible user interfaces (TUIs) is defined as user interfaces that allow the indi- 
vidual to interact with digital information through physical environments [47]. 
TUIs’ potential is to encourage exploratory and expressive activities to enhance 
learning [48], and there are several uses of TUI in educational settings [49,50]. 
TUI paradigm has been used for storytelling. For example, Ryokai and Cassell 
[51] developed Storymat, a play carpet that can record and replay children’s sto- 
ries to support children’s everyday narrative play. Instead, Shiva’s Rangoli [52] is 
an interactive and tangible storytelling installation that enables composing the 
emotional context of the stories by shaping the ambient settings (light, sound, 
music, and video). According to Gupta and colleagues [52], TUIs at the service 
of storytelling allows creating an emotional tone augmenting specific compo- 
nents like lights, sound, music to make them stand out more, aiding in narrative 
coherence. 

TUIs have been widely used to stimulate learning for children with special 
needs (for example, [53,54] and enhance participation, inclusion, and a sense of 
community in a classroom environment [55]. In this way, all students become 
constructive agents of their realities. 

Thanks to the TUIs approach, NEFELE wants to guarantee laboratory activ- 
ities for all children in the classrooms designed to create an environment that 
stimulates storytelling and design a sustainable and inclusive future for all. We 
believe that EDS can serve as a tool in educational systems to strengthen and 
foster the necessary skills as well as awareness of the choices available for one- 
self, to design and continuously improve a collective vision of multiple possible 
futures and ways of being both individually and collectively, to foster students’ 
optimism and hope about their future. 


6 Conclusions 


Adolescents career decision-making is required to go through a process of self- 
understanding, exploring an umbrella of career options with the aid of guid- 
ance and planning [56,57]. Life design interventions in a lifelong paradigm are 
intended to support students in their career plans, helping them generate actions 
that promote sustainable careers and decent working conditions [34]. 

EDS, through the creation of a digital story, makes the creator think more 
deeply [58]. We believe that this process is helpful to enhance student career 
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plans. As a tool for lifelong LD intervention, EDS is expected to foster students’ 
empowerment and fulfilment while tempering the impacts of fragmented career 
paths. 

In conclusion, EDS brings old and new and has an adaptable, dynamic and 
inclusive nature. Therefore, it allows it to be an optimal tool to empower students 
to shape their future. 
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Abstract. Due to COVID-19, higher education institutions all over the 
world transitioned to online learning. The sudden and forced transition 
to this new learning methodology pushed the Universities to rapidly ade- 
quate to the needs, upgrading their digital platforms to comply with the 
new requirements. In the same way, teachers had to adapt their teaching 
to fit the new medium’s potentials and limitations. The final receivers 
of this striking change, the students, had to adequate to the novelty 
approach, though this process has not been painless. Several difficulties, 
challenges and opportunities arose in this transition process for students, 
and the full digital class delivery also stressed them emotively. This study 
explores University of Foggia students’ perceptions of the emergency 
online learning. The factors analyzed involved their perception about 
the University implementation of the online class delivery, their consid- 
eration about the future of online learning and their emotional impact 
in attending courses with this methodology. Quantitative and qualita- 
tive data were collected from 3,140 participants. The findings present 
how students have globally appreciated online learning, but they mostly 
prefer a blended learning approach. Furthermore, by using segmentation 
variables, differences emerged among the participants’ groups, indicating 
online learning can have great potentials, but more complex and inte- 
grated approaches are needed to fulfill the different learning needs. 


Keywords: Online learning - COVID-19 - Students’ perception - 
University - Blended learning - Remote learning 


1 Introduction 


The world is facing a health crisis due to the COVID-19 pandemic. As a result 
of COVID-19, higher education all over the world has moved to deliver courses 
© The Author(s) 2022 
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online during Spring 2020 [2,7,12,21]. In particular, in Italy, online learning 
has been expanded to Fall 2020 and also Spring 2021 [3,11]. However, students 
have expressed stress related to online learning and difficulties compared to 
the traditional learning in physical classrooms [10]. By understanding students’ 
challenges and preferences, universities can develop strategies to assist students 
in case there are further weaves of COVID-19 or any other disaster requiring 
an emergency transition to remote learning. The shift towards online education 
during the pandemic of COVID-19 has led many studies to focus on perceived 
learning outcomes and student satisfaction in this new learning environment [1, 
4]. This paper explores Italian University students’ perceptions about online 
learning after COVID-19 government measures (stay-at-home and/or physical 
distance), and in particular students from University of Foggia. 

To reduce transmission of the COVID-19, several countries established mea- 
sures on infection prevention and control by limiting contact between people!. 
Governments suggested or ordered physical distancing and movement restric- 
tions”. 

Universities aimed to slow down the spread of COVID-19 by protecting all 
the individuals involved in the education compartments — students, staff, and 
faculty members — and to help ensure a safe and healthy learning environ- 
ment [5,13]. Many universities transitioned to remote learning where classes 
were held online [2,13]. Some universities were offering asynchronous classes 
where instructors prepare assignments or record lectures and students can com- 
plete them at their own pace [12]. Some institutions used synchronous learning 
that occurs at a specific time via a specific medium. In this paper, we focus 
on the case of the University of Foggia (Southern-Italy), a young University 
(about 20 years since its foundation) with about 11 thousand students in six 
departments (Table 1). The University of Foggia, even before the COVID-19 
pandemic, was engaged in online teaching. Specifically, 7 courses were in blended- 
form and the University of Foggia was leading the Eduopen project (http://www. 
eduopen.org/), a platform providing more than 300 massive open online courses 
(MOOGCs), involving 27 higher education institutions and 262 tutors and teach- 
ers. When the COVID-19 pandemic forced the University of Foggia (together 
with all the Italian education system) to switch to a complete online teaching, 
its previous experience helped this transition to be not so critical. The University 
of Foggia was already equipped with a Moodle platform, a Virtual Classroom 
Tool together with an e-learning center (founded in 2015) with specialized admin- 
istrative staff. As a result, the University of Foggia response to the COVID-19 
outbreak was not so dramatic. After the Italian President of the Council of 
Ministers’ decree®, enacted on 4 March 2020, all the activities were switched 
to a virtual form; lessons, exams, seminars, meetings were held online, and all 
planned academic activities have been confirmed, including graduation sessions, 
orientation events and research activities. In order to understand the effect of the 


1 https: //www.who.int /emergencies/diseases /novel-coronavirus-2019. 
? https: //www.cdc.gov/coronavirus/2019-ncov/index.html. 
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COVID-19 outbreak on the students’ community, the Italian National Agency 
for the Evaluation of Universities and Research Institutes (ANVUR — www. 
anvur.it) and the European Association for Quality Assurance in Higher Educa- 
tion (ENQA — www.enqa.eu) have promoted and established a working group to 
develop a survey to analyze the didactic experiences made in Italian universities 
during the COVID-19 health emergency, also in order to offer valuable elements 
in view of the strategies that the universities will have to adopt once the cur- 
rent pandemic phase will be over. In this paper, we analyze the questionnaire 
provided by the University of Foggia according to the indications received by 
ANVUR and ENQA. 


Table 1. Students at University of Foggia in academic year 2020-2021. Unique = 
Unique Cycle (Master degree), also known as Long First Degree Courses at Master 
level. 


Department Degree | Duration (y) | Year of enrollment 
I II \HI |IV|V |VI 
Economics Bachelor 3 539 |355|826 |- - - 
Master 2 167 |206|- - - - 
Law Bachelor 3 340 |235|655 |- - - 
Master 2 56 27 |- - - - 
Unique 5 154 |92 |103 |111|327 - 
Clinical and Bachelor 3 446 |129|207 |- - - 
experimental medicine Mäster 2 88 116 |- E _ _ 
Unique 6 51 41 |31 36 |28 46 
Agricultural science, Bachelor 3 244 |170|438 |- - - 
food, natural resources 
and engineering Master 2 207 |194|- - - - 
Medical and surgical Bachelor 3 273 |256|344 |- - - 
HAR Unique 6 101 101/80 |75 |76 269 
Humanities Bachelor 3 1253 | 655 | 1090 | - - - 
Master 2 283 | 335 |- - - - 


This paper is organized as follows: Sect.2 provides detail on the methodol- 
ogy adopted to survey students at University of Foggia. Section 3 presents the 
obtained results and discuss about them. Finally, Sect.4 concludes with final 
remarks and recommendations. 


2 Method 


To study the perceptions of online learning during COVID-19 pandemic at Uni- 
versity of Foggia, we built a questionnaire that circulated among students by 
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emails, messages, and word of mouth. Also University professors during lessons 
asked students to fill the questionnaire sharing the link. Such questionnaire 
includes questions to get preliminary information of participants (e.g., Uni- 
versity degree course they are enrolled in), questions about the perceptions of 
online learning due to COVID-19, and about preferences of learning modali- 
ties for future. More details are reported in Sect.2.1. Next, we analyzed the 
proposed questionnare. In this work, we select the Pairwise Markov random 
field method (PMRF), which relies also on graphs. Therefore, in this section, 
we briefly describe graphs in general (Sect. 2.2), and then how they are applied 
within the PMRF (Sect. 2.3). 


2.1 The Questionnaire 


In order to study the perceptions of online learning during COVID-19 pandemic 
at University of Foggia, we built the questionnaire reported in Table 2. The 
questionnaire is composed of 10 questions about perceptions related to online 
learning during COVID-19 (questions with Q identifier). To this we added fur- 
ther four questions asking participants general information (GQ identifier), their 
preferences on learning modalities for the future, and suggestions (LQ identifier). 
The answers to questions Qs are set on a 4-point likert scale (“Absolutely No”, 
“More No than Yes”, “More Yes than No”, “Absolutely Yes”). LQ is a nominal 
question and LQ2 is an open-ended question. Some of the questions (Q1, Q3, 
Q4, Q5, Q6, Q7, Q9, Q10) are written with a positive formulation: it means 
that a high agreement received as answer implicates a positive result. We refer 
to these questions as positive questions. Other questions (Q2, Q8) are written 
with a negative formulation: it means that a high agreement received as answer 
implicates a negative result. We refer to this latter group of questions as negative 
questions. 

We shared the questionnaire by emails, messages, and word of mouth starting 
from 1st April 2021. The last student participant submitted on 19 April 2021. 


2.2 Briefs on Graphs 


Network theory is the study of graphs, representations of relations occurring 
between discrete objects. Deeply rooted in graph theory — a branch of mathe- 
matics whose origins can be traced back to 1735, when the Swiss mathematician 
Leonhard Euler solved the Konigsberg bridge problem — network theory has been 
applied in disciplines spanning from statistical physics to computer science, from 
electrical engineering to biology or climatology [6, 14, 16-20]. In particular, as we 
will see in the following section, graphs can be applied to study questionnaires. 

Graphs are very abstract conceptual structures that can be used to model 
relations and processes taking place in extremely different systems. A network is 
a graph with N nodes (or vertices) and L links (or edges) that can be weighted or 
unweighted, directed or not. An unweighted network is completely represented 
by its N x N adjacency matrix A such that A;; = 1 if node i points to node 
j, Aij = 0 otherwise. Let G = (V, E) be a graph, where V is the set of its 
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Table 2. List of the questions (Q) about online/remote learning posed to students at 
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University of Foggia. 


Identifier | Question (English ver.) Original question (Italian ver.) 

GQl To which University degree course you A quale corso di laurea sei iscritto? 
are enrolled in? 

GQ2 To which kind of degree course you are A quale tipologia dicorso di studi sei 
enrolled in? iscritto? 

GQ3 To which year of the degree course you A quale anno del corso di studi sei 
are enrolled in? iscritto? 

GQ4 How far do you live from University of Quanto distante vivi dall’Universita 
Foggia? degli Studi di Foggia? 

Q1 Do you think the e-learning platform is Ritieni che la piattaforma e-learning 
easy to use? sia di facile utilizzo? 

Q2 Have you struggled to find the teaching Hai riscontrato difficoltà nel reperire il 
material (textbooks, handouts, slides, materiale didattico (libri di testo, 
etc.)? dispense, slide, ecc)? 

Q3 With remote learning, has your class Con la didattica a distanza, é 
attendance increased (synchronous aumentata la tua frequenza alle lezioni 
and/or asynchronous)? (in sincrono e/o in asincrono)? 

Q4 Do you believe that listening to the Ritieni che l’ascolto delle registrazioni 
lessons recording on the e-learning delle lezioni presenti sulla piattaforma 
platform are useful for the purpose of e-learning siano utili ai fini dello studio 
studying and learning the concepts e dell’apprendimento dei concetti 
explained? spiegati? 

Q5 Did the teachers mostly manage with I docenti hanno prevalentemente gestito 
ease the tools/platform used for remote con facilità gli strumenti/la piattaforma 
learning? utilizzati per la didattica a distanza? 

Q6 Did the equipment (computers, tablets, Le apparecchiature (computer, tablet, 
smartphones) and the internet smartphone) e la connessione internet 
connection available to you allow you to | a tua disposizione ti hanno permesso di 
enjoy in a satisfactory way (audio, video, | fruire in modo soddisfacente (audio, 
interactivity) the activities of the remote | video, interattività) delle attività degli 
learning courses? insegnamenti erogati a distanza? 

Q7 Are you overall satisfied with how the Sei complessivamente soddisfatto di 
online or blended teaching was carried come si è svolta la didattica online o in 
out for the courses of your Degree modalità duale per gli insegnamenti del 
course? tuo Corso di Studio? 

Q8 Did the lessons in online mode improve La fruizione delle lezioni in modalita 
your time management? online ha migliorato la gestione del 

tempo a tua disposizione? 

Q9 Do you think that the remote learning Pensi che la DaD ti consenta una 
eased the communication with the maggiore facilità di interlocuzione con 
teacher, since you can express your il docente, poiché puoi esporre le tue 
opinions without the embarrassment of opinioni senza l'imbarazzo di parlare in 
public speaking and/or the fear of being | pubblico e/o il timore di essere 
judged? giudicato? 

Q10 Do you think that remote learning Pensi che la DaD influenzi 
negatively affects your emotional state negativamente il tuo stato emotivo 
(anxiety, stress, sense of isolation and (ansia, stress, senso di isolamento e 
loneliness, etc.)? solitudine, ecc)? 

LQ1 At the end of the COVID-19 emergency | Al termine dell emergenza COVID-19 
what type of teaching method would you | quale tipo di metodo di insegnamento 
choose? sceglieresti? 

LQ2 Further suggestions/observations Ulteriori suggerimenti /osservazioni 
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vertices such that |V| = N and E is the set of its edges such that |E| = L. 
Edges may denote just the connection among two nodes or being labeled with 
a number indicating weights assigned to them. In the latter case, the graph is 
called weighted. 


2.3 The Pairwise Markov Random Field Method 


Pairwise Markov random field (PMRF) is a well-known network model, used for 
estimating psychological networks. The PMRF are networks where the nodes 
represent variables connected by undirected networks, i.e., edges where the rela- 
tion between nodes can be traversed by both ways. The edges between the nodes 
(variables) indicate that there is conditional dependence between the two vari- 
able, while low weight or absent edges indicate that the two variable are inde- 
pendent after conditioning on other variables [9]. When data are multivariate 
normal, such a conditional independence would correspond to a partial correla- 
tion being equal to zero [9]. According to [8] this method can be applied also on 
ordinal data, which are available in the dataset we gathered for this research. 
We can imagine a network model G = (Y, E) with three nodes y1, y2, y3 and 
undirected edges (y1, y2) and (y2, 43). We can consider yı conditionally indepen- 
dent from y3, given yg that may be a mediator or a moderator in a network 
psychometric. But if we remove the y2 from the network, the two variable may 
be correlated. The two variable yı and y3, may be the answers to the questions 
“Are you happy to go to work?” and “Do you consider your work environment 
comfortable?”, the two question have a high chance to be correlated, but their 
correlation could be explained by the variable yo “Is it your dream job?”. 

When the data follow a multivariate normal density, the appropriate PRMF 
model is called the Gaussian graphical model (GGM; [15]), in which edges can 
directly be interpreted as partial correlation coefficients [9]. 

Such network is encoded in a symmetrical and real valued px p weight matrix 
Q, which elements w;x represents the edge between the node j and k given by 
Cor(y;, yy #9) = wjp = WE; Where y~%*) is the vector of variables excluding 
the variable y; and yx [9]. The partial correlations coefficient can be obtained 
from the inverse of variace-covariance matrix X, also named the precision matrix 
K [15]. In case of ordinal data are used polychoric correlations [8]. 

The GGM model used is defined by: 


>= A(I- QUA (1) 


where A is a diagonal matrix with 46;; = k7? and Q has zeros on the diagonal. 

The edge missing between nodes means the two variable are conditionally 
independent [9]. 

In this work, we used GGM as a way to visualize the correlation between 
different segments of the sample we used. In Fig.7 the tree different models 
show the variable interaction, and are helpful to explain what factors are more 
connected by the typology of students. 
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The GGM model have been generated using the R package “psychonetrics”4. 
Figure 10 shows the most used keywords in the conclusive open question 


“Further observations”, and is built with the R package “tm”®. 


3 Results 


In this section, we report the results obtained from the analysis of question- 
naire. We gathered a total of 3,140 participants, i.e., 26.5% of total students 
ad University of Foggia. The analysis is split as follows: Sect.3.1 presents the 
result obtained analyzing GQs; Sect.3.2 shows the results obtained studying 
Qs through both descriptive and PMRF analysis; lastly, Sect. 3.3 illustrates the 
future preferences of students with respect to learning modalities and the anal- 
ysis of their suggestions. 


3.1 Analysis of GQs 


In this section, we analyze the questions GQs (Table 2). Information about GQ2, 
GQ4 are reported in Table 3. To ease the reading, results of GQ1, GQ3 are 
reported in Fig.3 and Fig.4, respectively. 

The distribution of participants with respect to total students per degree 
course is shown in Fig.la. We observe that, except for Humanities and Law 
(20% or lower), all degree courses provided a similar percentage of participants 
(more than 25% and less than 35%). In Fig. 1b, we show the distribution of 
participants with respect to total students at University of Foggia based on the 
year of enrollment. We notice that the majority of students of the last years were 
involved as participants in the questionnaire, with respect to what happens for 
the first three years (less than 30% of students participate to the questionnaire). 
Figure 2 also show the distribution of student’s enrollment year by course type, 
indicating a great part of respondents are enrolled at first year of a bachelor 
course. 

The distribution of participants by degree course is depicted in Fig.3; the 
participants were equally distributed across the different courses at University of 
Foggia (from 14% of Clinical and Experimental Medicine, to 22% of Humanities). 

The participants were mainly enrolled in Bachelor degree courses (66.3%); 
the rest are evenly split between Master degree courses and Unique Cycle degree 
courses, 17.8% and 15.7% respectively (Table 3). The vast majority of them are 
enrolled in the first and second year of University (63% of total participants). 
We gathered answers from a small percentage of participants from four, fifth, 
sixth and successive years (Fig.4). Furthermore, the vast majority of them do 
not live in the city of Foggia: in particular, 37.8% of them live within 50 km from 
University of Foggia, while 33% live more than 50km far from the University 
(Table 3). 


4 https: //cran.r-project.org/web/packages/psychonetrics/psychonetrics.pdf. 
5 https: //cran.r-project.org/web/packages/tm/tm.pdf. 
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Fig. 1. Distribution of participants to questionnaire with respect to students at Uni- 
versity of Foggia. 


Participants per type of degree course and year of enrollment in the specific course 


=" Bachelor = Master = Unique Cycle (Master) 


U 


IV year 


VI year 
and 
successive 


Fig. 2. Participants to questionnaire split based on the type of degree course and the 
year of enrollment to that specific course. 
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Fig. 3. Distribution of participants per Department at University of Foggia. 


Table 3. Distribution of students by course degree and distance from Foggia 


GQz2 Typology of degree course 
Bachelor 0.663 IE 
Master 0.178 W 
Unique Cycle (Master) 0.157 I 


GQ4 Distance from University of Foggia 


Foggia 0.288 I 
<50Km 0.378 I 
>50Km 0.330 I 


3.2 Analysis of Qs 


The analysis of Qs (Table 2) is split into descriptive and PMRF analysis. In the 
following, we first dwell on the descriptive study and then on the one conducted 
with PMRF method. 


Descriptive Analysis. For this analysis we have chosen boxplots. Boxplots are 
an effective way to display results, comparing the distributions of samples. The 
boxplot uses as main information for the structure of the graphic, the median 
and quartiles. The median is represented in the graphics of this work with the 
red line. The upper and lower limits of the white box represents respectively, 
the thid quartile and fist quartile. Thus, in the white box falls the 50% of the 
distribution. The whiskers encompass the values falling in a range of 1.5 of the 
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GQ3 Participants per year of enrollement 
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Fig. 4. Distribution of participants per year of enrollment at University of Foggia. 


interquartile range (third quartile minus first quartile) out of the upper and 
lower margin of box. Values outside this range are represented a outlier points. 
In Fig. 6, we display the boxplots answers for questions Qs based on partici- 
pants home distance from University of Foggia. In particular, we split between 
those living in Foggia city, those living within a radius of 50 km from Foggia, and 
the ones living more than 50km far from Foggia (including foreign students). 
Overall, the figure shows how the majority of questions have received posi- 
tive answers towards remote and blended learning (according to the positive and 
negative formulation of the question) by all three groups of participants. The 
most extreme results come from the question Q4, about the utility of recorded 
lessons, and secondly from question Q1, about the easiness of e-learning plat- 
form. Questions about the increased class attendance (Q3), Instructor profi- 
ciency for remote teaching (Q5), and possession of adequate devices and con- 
nection (Q6) show a wide positive accordance among groups. Few students had 
difficulties in finding didactic material (Q2) and all the three groups showed a 
high overall satisfaction for remote teaching (Q7). While all three groups experi- 
enced a improvement in time management (Q8), as we could expect the answers 
are more positive for students living outside Foggia city. The reasonable explana- 
tion is that these students have spend more time traveling to Foggia with tradi- 
tional learning. This gap between students living in Foggia city and other cities 
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Fig. 5. Participants’ answers by their home distance from University of Foggia. Foggia 
= Students that live in the University’s town, <50 km = students living within a radius 
of 50 km from University of Foggia, >50km means students living more tha 50 km far 
from University of Foggia. 


is substantially suppressed by remote learning. The question Q9, about better 
conversation with teacher, shows a higher variance among groups. In particu- 
lar, students living in Foggia city perceived a worsening of interaction “quality” 
with teachers during COVID-19 pandemic, while students outside experienced 
an improvement. Speculation about this difference may settle in the facts stu- 
dents from Foggia city always had better opportunity to attend class dialog with 
teachers directly, while students from outside experienced more limitation due to 
public transportation time constrictions and generally more difficulties in attend- 
ing classes. The question Q10, about the negative impact on the emotional state, 
also highlight that students living in Foggia city were more negatively impacted 
from remote learning. The reasons may lie in the advantages and easiness of 
reaching the University facilities and higher opportunities in establishing rela- 
tionship with other students. 
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Fig. 6. Participants’ answers by typology of degree course. Unique = Unique Cycle 
Master course. 


In Fig. 6, we display the answers boxplots for questions Qs based on which 
typology of degree course the participants are enrolled in. In particular, we split 
between Bachelor, Master, and Unique Cycle (Master degree). In red, we depict 
the median. Overall, the figure shows how the majority of questions have received 
positive answers towards remote and blended learning (according to the positive 
and negative formulation of the question) by all three groups of participants. 

The most extreme results come from the question Q4, about the utility of 
recorded lessons, and secondly from question Q1, about the easiness of e-learning 
platform. Questions about the increased class attendance (Q3), Instructor pro- 
ficiency for remote teaching (Q5), and possession of adequate devices and con- 
nection (Q6) show a wide positive accordance among groups. Few students had 
difficulties in finding didactic material (Q2) and all the three groups showed a 
high overall satisfaction for remote teaching (Q7). While all three groups expe- 
rienced an improvement in time management (Q8), Master’s students showed 
a more significant improvement. Possible explanation settles in the long expe- 
rience of such students in managing studying activities and because they often 
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have a job or internship. Question Q9, about better conversation with teacher, 
shows a high variance among groups. In particular, Bachelor students perceived 
a worse kind of interaction with teachers during the COVID-19 pandemic. The 
main reasons for this difference maybe they are less experienced and new in the 
University environment, and the bachelors’ courses are usually more crowded, 
affecting more the possibility of a fruitful interaction on an e-learning platform. 
Lastly, Q10 indicates that Bachelor students had the most negative impact on 
the emotional state. This is not unexpected, considering that most of the partici- 
pants in our questionnaire were enrolled on the first year (Fig. 4). These students 
experienced the shifting from High schools to University during the COVID-19 
emergency and subsequent restrictions. Therefore, they had few opportunities 
to settle in the environment and establish a relationship with other students. 


PMRF Analysis. Figure7 and Fig.8 shows the graphs created according to 
the methodology described in Sect.2. The edges in the graphs indicate the pres- 
ence of a connection between answers of the questions, over the threshold of 
0.15, empirically chosen as equilibrium point between informativity and legibil- 
ity. The thicker the edges, the stronger is the connection between the variables. 
The blue edge color indicates a direct connection between the questions, i.e., a 
higher(lower) score of question A matches a higher(lower) score of question B. 
Instead, the red edge color indicates an inverse connection between the nodes, 
i.e., a higher(lower) score of question A matches a lower(higher) score of ques- 
tion B. A particularly prominent example of inverse connection in our work is 
the relation between node Q8 and Q10. Q8 asks the student to provide a higher 
score when remote learning improved his/her time management — a positive fea- 
ture for remote learning. On the other hand, Q10 asks to provide a higher score 
if the remote learning negatively affected her/his emotional state — a negative 
feature for remote learning. Although the two questions ask about two different 
features of remote learning, some sort of connection between them is reasonable, 
as the emotional state can impact time management and vice versa. The way the 
question has been asked imply that if students are in a negative(positive) atti- 
tude toward remote learning they (should) give a lower(higher) score to Q8 and 
higher(lower) score to Q10. In doing this, participants create an inverse connec- 
tion reflected trough the red edges in the graph. One or more latent factors likely 
impacted the student’s answers to the questionnaire; however, the questionnaire 
has not been structured to be managed with proper methodologies relative to 
the detection of the latent factors. 

In Fig. 7 the three graphs have been generated filtering questionnaires based 
on where the participants declared to live. International students have been 
merged with the group of students living more than 50 km far from the Univer- 
sity of Foggia. This segmentation is because students from Foggia have reduced 
problems in getting to the University facilities. Instead, students living in a town 


6 High schools classrooms are usually smaller in terms of students (15-30); therefore 
University newcomers are more used to face-to-face conversations with the teacher 
than what happens at University. 
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within a radius of 50km are likely to be commuters and have more difficulties 
reaching the university facilities, as they need to spend more time traveling. The 
last group are the students coming from town distant more than 50km. These 
are likely to be off-site students, with typical challenges of young students living 
in other cities far from relatives and hometown-friends. 

The node numbers in the graphs match the questions of Table2 and the 
features reported in Table 4. In all three graphs of Fig.7, a strong connection 
between some features is evident. For example, the edge between Q5 and Q7 
shows a connection between the score of Instructor proficiency for remote teach- 
ing and the overall satisfaction for remote learning. The edge between Q3 and 
Q8 indicates a strong connection between the increased class attendance and 
the time management improvement as expected. Nodes Q8 and Q9 indicate a 
connection between the improvement in time management and the conversation 
with the teacher. The edge between Q8 and Q10 indicates an inverse connec- 
tion between time management improvement and the negative impact on the 
emotional state. Likewise, the edge between Q9 and Q10 indicates an inverse 
connection between improvements in conversion with the teacher and the nega- 
tive impact on the emotional state. Figure 7b and 7c shows commuters and offsite 
students have a higher inverse connection between Q7 and Q10, i.e. the overall 
satisfaction for remote and blended learning and the negative impact on the emo- 
tional state, and also a negative correlation between Q2 and Q5, i.e., difficulty of 
finding didactic material and Instructor proficiency for remote teaching. For stu- 
dents living more than 50 km far from University of Foggia (Fig. 7c) there is also 
a stronger connection between Q8 — time management improvement — and Q3 — 
increased class attendance. Moreover, Q3 is also inversely connected with Q10 — 
the negative impact on the emotional state. These connections may be because 
many students very distant from university do commuters or often do not attend 
class. Students living within a radius of 50 km from the University of Foggia in 
Fig. 7b, shows a higher connection between Q1 — easiness of e-learning platform 
—and Q5 — Instructors proficiency for remote learning —, and also a connection 
between Q3 — increased class attendance — and Q7 — the overall satisfaction for 
remote learning. Students living in Foggia in Fig. 7a, showed a higher connection 
between Q4 and Q7, utility of recorded lessons and the overall satisfaction for 
remote learning, and between Q7 and Q9 — better conversation with the teacher. 
Both students living in Foggia and a town within a radius of 50 km from Foggia 
showed an inverse connection between the easiness of the e-learning platform 
and the difficulty of finding didactic material. 

Figure 8 shows the three graphs generated filtering questionnaires based on 
the type of course attended by students. We can expect the students attending 
a bachelor are less experienced than Master degree students. They are younger, 
and usually, they just come directly from high schools. On the other hand, master 
degree students are more experienced in the university environment and probably 
more used to the University of Foggia itself. Finally, there is a more complicated 
category, i.e., Unique cycle students, that encompasses new students and older 
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Table 4. List of feature measured with the questions (Q) about online/remote learn- 
ing posed to students at University of Foggia. The exact corresponding questions are 
indicated in Table 2 


Identifier | Feature 

Ql Easiness of e-learning platform 

Q2 Difficulty of finding didactic material 

Q3 Increased class attendance 

Q4 Utility of recorded lesson 

Q5 Instructor proficiency for remote teaching 

Q6 Possession of adequate devices and connection 
Q7 Overall satisfaction for remote and blended learning 
Q8 Time management improvement 

Q9 Better conversation with the teacher 

Q10 Negative impact on the emotional state 


(b) Students living in a city (©) 
(a) Students from Foggia. within a radius of 50km from 
Foggia. 


Students living more 
than 50Km far from Foggia 


Fig. 7. Relation between the questions based on participants home distance from Uni- 
versity of Foggia. Nodes represent questions (Q) — see Tables2 and 4. Blue edges 
indicate a direct relation between the questions, red edges indicate an inverse rela- 
tion between the questions. The thicker the edges the stronger relation between the 
questions. (Color figure online) 


ones. However, these students are mainly enrolled in the program of Law and 
Medical and Surgical Sciences. 

The consideration above pushed to explorer if these differences could affect 
the connection of the question. Many of the general considerations about the 
connection between questions made above for Fig.7 still apply in this case. In 
particular, it mainly holds true for the nodes Q5, Q7, Q8, Q9 and Q10. However, 
it is relevant to shed light on some relations for Master students in Fig. 8b. There 
is a stronger connection between Q6 and Q7, i.e., possession of adequate devices 
and connection and the overall satisfaction for remote learning, deserving deeper 
future investigation, and inverse connection between Q2 and Q5, i.e., difficulty 
of finding didactic material and Instructor proficiency for remote teaching. Par- 
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ticularly interesting is the network of students enrolled in a Unique cycle Master 
program depicted in Fig.8c, which shows more edges than the others. For exam- 
ple, the edge between Q1 and Q5, highlight a connection between the easiness of 
the e-learning platform and the Instructor proficiency for remote teaching. The 
edges between Q3 and Q10 indicate an inverse connection between increased class 
attendance and negative impact on the emotional state, while the edge between 
Q5 and Q8 also shows an inverse connection between Instructor proficiency for 
remote teaching and time management improvements. Lastly, there is a connec- 
tion between Q4 — utility of recorded lessons — and QT — overall satisfaction for 
remote blended learning. 

It is worth highlighting that the edges shown in this section’s graphs have 
to be interpreted only in terms of connection and not as indicators of a more 
positive (or negative) factor for one students’ group with respect to the others. 


(a) Bachelor (b) Master (c) Unique cycle (Master) 


Fig. 8. Relation between the questions based on participants’ type of course. Nodes 
represent questions (Q) — see Tables 2 and 4. Blue edges indicate a direct relation 
between the questions, red edges indicate an inverse relation between the questions. 
The thicker the edges the stronger relation between the questions. (Color figure online) 


3.3 Analysis of LQs 


In this section, we analyze the two LQs (Table 2). 

The LQ1 question asked students what type of teaching methodology would 
they choose in future. 

Figure 9a shows the distribution of participants answers by where they live. 
For students living in Foggia city, the most relevant answer are Traditional learn- 
ing only and Bledended learning, with more than 5 but less than 2 of lessons. In 
contrast, students living within a radius of 50 km from Foggia prefer Bledended 
learning, with more than 5 but less than 2 of lessons, and students living more 
than 50 km far from Foggia prefer Online learning only. The different propensity 
of the three groups can be easily justified by the difficulties these students face 
in reaching the University facilities: clearly, this impacts more students living 


outside Foggia. 
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LQ1 At the end of the Covid-19 emergency what type of LQ1 At the end of the Covid-19 emergency what type of 
teaching method would you choose? (by degree course) teaching method would you choose? (by home distance) 
100% 100% 
90% 90% 
80% 80° 
70% 70% 
60% 60% 
50% 50% 
40% 40% 
30% 30% 
20% 20% 
10% 10% 
0% 0% 
Master Unique Cycle (Master) Bachelor >50km <50Km Foggia 
m Traditional learning only # Online learning only m Traditional learning only m Online learning only 
= Blended learning* Blended learning** m Blended learning* Blended learning** 
(a) By typology of degree course. (b) By distance from University of Foggia. 


Fig. 9. Participants’ preferences for the future learning approach. * More than 4 but 


less than 2 of lessons and other teaching activities are held in traditional learning 
modality; the rest are held online. ** More than 2 of lessons and other teaching 
activities are held online. 


Figure 9b shows the distribution of participants answers by the typology of 
degree course they are enrolled in. The preferred answer for all the three groups 
is Bledended learning, with more than b but less than 2 of lessons. For Mas- 
ter students, there is also a strong preference for Online learning only, while 
Unique cycle Master students highly prefer Traditional learning only. Overall, by 
analysing the results obtained, blended learning approaches are the more promi- 
nent among all groups. The predilection of master students for online learning 
may be justified by the need to conjugate study with a job. 

LQ2 is an open-ended question, inquiring students to provide further obser- 
vations. Five hundred students answered this question with comments whose 
length varied from just one word to long comments. Figure 10 shows the word 
cloud of the top-100 words used by students in their comments on this ques- 
tion. The comments cover different areas and topics. Many of them contain an 
appreciation for remote learning (in particular for recorded lessons). Such com- 
ments come mainly from commuters and students with a job. They stated they 
benefited from remote learning and wish to continue to benefit in future. 

Other comments focus on appreciation for the e-learning platform and how 
the University of Foggia managed it. On the other hand, others complain about 
issues experienced with the platform. Moreover, some comments provided sug- 
gestions to improve the management and experience of remote learning. Lastly, 
several students manifested discomfort for remote learning approach, complain- 
ing about the lack of interaction and relationship with students, apathy and 
alienation of attending class from a device, anxiety and discomfort in taking 
exams online, and unsuitability for matters requiring practical assistance expe- 
rience. 

In summary, the comments suggest that remote learning may be a viable and 
appreciated way to deliver classes but does not fit all the students’ needs. 


On the Perceptions of Online Learning Due to COVID-19 Pandemic 147 


connessione 


o 
personalmente az 25 
situazione = = 55 spesso 
= Coe = p 
possibile parte SgS E dope 
5 ì 
svolgere Quindi alcuni $ 3 utile così 


avere S modo professori permette 


corso gg 
tomas © o B° didattica problemi 
© . 
rengo E 2 5 tutte seguire docenti continuare 
Soil 3 RI 
2es58 o 7E Pae ace È 
SEY a O solo aa 28 
oe 224 D © © 
eg O OH E = 
S84 5 © meglio © a 
5 O OOE n 
cosa a pa (e) pgn 
È possibilità fare ‘= tempo & È 
z P ad = aS 
—_ j (0) oc 
smpe modalità aE ® lezione è H 
Š È O oO 
invece inoltre distanza © DD ae £ cova z 3° 
i 9 O & z studente 
difficoltà stato online =% 5 © 
: inni g0 = poter $ volte 
possa registrazioni § Gp 2 
molti Grazie tirocinio 89  s 
2 2 
momento registrazione orsi S S 5 n 
O 


studiare registrare 
preferisco attività 


Fig. 10. Wordcloud of the top keywords (uni-grams) used in answers to the open-ended 
questions concerning “Further observation” in the questionnaire. 


4 Conclusion 


In this work, we performed an exploratory analysis of the impact of remote and 
blended learning methodologies due to COVID-19 on the students’ community at 
the University of Foggia. For this reason, we surveyed students perceptions with 
a questionnaire whose items aimed to provide to the University a feedback about 
the quality of the implementation of the e-learning platform and teaching, how 
these modalities have been accepted by students and the students’ propensity 
for the next future to the different learning approaches. 

The analysis showed how the overall opinion of students about the experience 
offered is positive. However, despite the global feedback, the analyses showed 
how different students’ segments had slightly or substantially different experi- 
ences. In particular, students living outside Foggia city have benefited a lot from 
remote learning, while students living in Foggia experienced more difficulties and 
reduced human contact with the online learning approach. These aspects have 
been also confirmed by the different preferences these segments expressed about 
the future learning modality at University. 

Moreover, the students enrolled in Bachelor programs had more difficulties 
integrating into the university environment due to their direct transition from 
high school to University with e-learning and COVID-19 restrictions. 
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The Pairwise Markov random field analyses provided insights on some pecu- 
liar interactions between the measured factors specific for some segments of 
students. However, this analysis has been limited by the shortcomings of the 
questionnaire adopted, with too limited values of Likert items, and lacking 
some crucial questions to better segment the students and more fine-grained 
questions targeting students’ psychological experience. On the other hand, the 
insights gained through this survey and the speculation inherent some exciting 
results provide the foundation to elaborate more sophisticated research, aiming 
to answer some of possible emerged research questions. 

In summary, the data lead us to consider that the implementation of remote 
learning at the University of Foggia was successful, despite some difficulties. 
However, the appreciation and the emotional experience diverges across different 
segments of students. This means that remote learning cannot be the ultimate 
solution to fit the complexity of learning needs, but more tailored programs and 
learning strategies have to be implemented to cope with the needs of different 
students at the University of Foggia and in Italy in general. 
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